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Preface 



Large cross-national surveys of educational achievement take a decade to plan and carry out, involve 
thousands of people and cost millions of dollars. Why do people do it? What benefits could possibly justify 
such an effort? This report is one attempt to answer these questions, an answer given in the context of one 
of the two Canadian provinces that participated as "countries" in the Second International Mathematics 
Study (SIMS). The project was carried out in the early 1980s under the aegis of the International 
Association for the Evaluation of Educational Achievement, better known as lEA. It was one of an ongoing 
series of such surveys, including mother tongue and foreign languages, science, classroom environment, 
schcol-to-work snd item banking. 

The first international mathematics study was completed in the 60s and the results reported in two 
volumes (Husin, 1967). That study concentrated on school organization, including curriculum, using 
achievement in mathematics as a general indicator of school success. It was not a study of mathematics 
education, but rather a study of schools and schooling, using mathematics achievement as an indicator of 
output. 

From the beginning, the second study (SIMS, as it came to be known) was planned as a 
documentation and analysis of mathematics education. Mathematics achievement was only one 
component, albeit an important one, of the overall picture. The objective was to learn about the teaching of 
mathematics-what was taught, how it was taught and what methods were most successful. By comparing 
and contrasting the findings from small and large countries around the world, both developed and 
developing, greater understanding can be reached tJian if studies are done separately in one country at a 
time. School organization was documented and social class measured as correlates of the curriculum and 
teaching methods as well az of achievement. 

Two important iransition points were chosen for the mathematics education stndy-the earlier one 
at the end of elementary school or the very beginning of secondary and the later one at the end of secondary 
school. The sample at the earlier point. Population A for short, was to be taken from all students in the 
grade in which the largest number had attained the age of 13.0 - 13.11 years of age by the middle of the 
school year. In most countries, ah children are still in school at this age. In contrast, the sample at the 
later point included only students who were specializing in mathematics-those who were studying 
mathematics for five or more hours per week. By this time, the proportion of young people still in school is 
under 10 percent of the age group in some countries. A list of participating countries and the abbreviations 
to be used for country names is given in the display below. More information on participating countries is 
presented in Appendix A. 

As this little book is being written, most of the participating countries have published their 
"national" reports, and the results of the international analyses are in press. Numerous topical reports 
and S-' eral overviews have already appeared in teachers' publications and in academic journals.^ 

Chapter 1 reviews the design of SIMS and describes how different interpretations of the two 
populations led to quite different samples of students from country to country. Ontario's Population A 

ERIC ^ 



Countries Participating in the Second International Mathematics Study 
and Abbreviations Used in This Eaport 



Belgium, Flemish schools (BPL) 
Belgium, French schools (BPR) 
Canada, B.C. (CBC) 
Canada, Ontario (CON) 
England and Wales (ENW) 
Finlend(FIN) 
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Hong Kong (KKO) 
Hungary (HUN) 
Ireland (IRE) 
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Japan (JPN) 
Luxembourg (LUX) 
Netherlands (NTH) 
New Zealand (NZE) 
Scotland (SCO) 
Swaziland (SWA) 
Sweden (SWE) 
Thailand (THA) 
United States (USA) 
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sample (taken from Grade 8 classes) was more varied than most and more representative than British 
Columbia's, for example. The older group (Population B) represented a wider segment of the Ontario 
population than in most countries, even though the definition limited the group to students taking at least 
two of the three advanced mathematics classes at the Grade 13 level. Ontario's diverse population also 
showed itself in an unusual range of both parents' occupation and education. 

The second chapter is concerned with the officially mandated curriculum, that specified by the 
Ministry of Education in curriculum guidelines and with the assembly of items to test achievement of 
students who study within those curriculums. Ontario is not so different from the rest of the world. 
Teachers everywhere make many choices, however, within the guidelines, and their preferences result in 
important variations from school to school and from class to class in' what is taught and what is 
emphasized. Characteristics of Ontario schools and teachers are described in this chapter and compared 
with schools and teachers in other countries. 

Ontario student achievement and attitudes are examined in Chapter 3. By this time. Chapters 1 and 
2 have made it clear that there are no simple, direct comparisons to be made among these different types of 
samples from different countries. This is true even between Canada (Ontario) and Canada (British 
Columbia)! Achievement is therefore discussed first in Ontario, topic by topic, often item by item, in 
relation to other characteristics. In view of the widespread interest in gender diff^erences, achievement 
.differences between boys and girls are explored. Everywhere, achievement is related to instructional 
emphasis, that is, to students' opportunity to learn. Attitudes are regarded as an outc^ome in their own 
right as well as a measure to be correlated with achievement. On average, the performance of Ontario 
studer ts placed them at the middle of participating countries, Ontario often defining the exact median 
among countries. 

The final chapter is a presentation of some of the lessons learned and plausible conclusions from the 
study. Results from Ontario ate discussed mainly in the international context. Given that the 
international volumes are not even published yet, it is too soon for any summary of conclusions. That will 
take still more study and perhaps further analyses. It does seem that Ontario offers less mathematics in 
Grade 8 than some other countries, but more mathematics is offered to more students in Grades 12 and 13. 
Many countries offer more algebra in Grade 8, for example, and students show they can learn the content. 
Lots of time is spent everywhere reviewing old arithmetic content, but no systematic gain in achievement 
results from it. Most countries channel students into several tracks by the time they reach Grade 8, 
Ontario, Prance and Japan being exceptions. 



rne amount oi nomework done varies a great deal from country to country. Students who do more 
homework tend to achieve slightly higher scores, as most people expected. There may be some question 
whether the amount of calculus learned justifies the amount of time spent on it. Ontario has a high rate of 
student participation in Population B, compared to other countries, and the top students compare very well 
with top students elsewhere. In the end, however, there is no one conclusion. There are many conclusions 
and much feed for thought. 
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Chapter 1 

The Many Faces of Mathematics Education 



The first international mathematics study was a pioneering effort and the reports have been widely cited 
The study was also widely criticized, however, especially in the mathematics and mathematics education 
communities. A particularly lengthy and trenchant criticism came from a professor in the Netherlands 
Who was at the time the editor of a respected mathematics education journal (Freudenthal, 1975) He 
attacked virtually every aspect of the study and aoted that mathematicians and mathematics educators 
had not been centrally involved in its planning or implementation. 

When a group met in Scotland in 1974 to consider undertaking the second study, they resolved that 
both mathematicians and mathematics educators would be given prominent place. The first international 
meeting to consider details was held at the University of Illinois in 1976, where they began to design a 
study of mathematics education that would document what was being taught and how it was being taught 
m addition to what was learned. In addition, there would be a thorough survey of school system, school 
teacher and student characteristics that would enable interpretation of the teaching and learning findings 
m a wide context Teacher and student attitudes would also be recorded in detail, ail to be done at the end 
of elementary school (Population A) and at the end of secondary school (Population B) to fill out the most 
complete picture of teaching and learning yet attempted for any subject. 



1.1. Describing the Content and the People 

In order for SIMS to be a study of mathematics education and not simply a survey of student achievement 
lIZ^T ^^t.""^'^.'^' °f '^-ching and learning that was amenable to study' 

by testing, questionnaires and analysis of documents. Provision was made for achievement testing at the 
beginning and at the end of the school year, referred to as the pretest and the posttest. Only eight 

Tmtut hTT ^" ^' only one (the uIa) 

administered both in Population B. A whole series of questionnaires were developed for the other 
dimensions of the study, designed for administration at several times during the school year. The various 
torms and test booklets and their timing are illustrated in Figure 1-1. 

.hp J^r^'%^'u '^""^""""^^'^^ '''' ^^'^^ lis^^d under Pretest were the ones administered at 
•stedTr '"n" ^'^'^ "° ^^^'^^^ ^" - ^-^^ ^2 and 13. The forms 

olnr t1^^7°'".k '^"""'^''^ J'"^' 'hey finished teaching the 

ZThL I i t "I . "■"'''^ questionnaires is illustrated in Figure 1-2 by a selection of 

aThTrsib? T : I". " " '''''' have heard of 

all .he possible ways to teach fractions, and indeed a few tended to dominate. Each such form asked about 

Z t^tTl Tr?' °" interpretations of important concepts 

and opinions about teaching ths topic. ^ 
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Figure 1-1: School, teacher and student questionnaires and test booklets 
used with Populations A and B in Ontario's implementation of the 
second international mathematics study. 
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for the topic fractions 



Teaching Methods - Common Fractions 
The interpretations of fractions given below may be included in your instructional program. 



FractionB as parts of regions: 
^ aeani 




FractionB at parts of a 
collection: 




Fractions as the coordinates 
of points on a number line: 

^ 1 i I I I I I ♦ I I I i ; 

-1 0 



Fractions as quotients: 
^ means "3 divided by 

Fractions as decimals: 

3 



0.T5 



Fractions as repeated addition 
of a unit fraction: 

^^^^ 

Fractions as ratios: 
^ Boans • • • 

AAA A 

Fractions as aaMurviVTits: 
this eo«tai«er holds 



3 

7 L 



6 



this stick la 7 cm 
Fractiooa as operators: 



Fractions as comparisons: 



J unit rod 



3* 
2 

"3 ^ 



1 2 
T»fO -J rods ■ a ^ rod 



mm 



i decimal as the coordinate of a 
point on the number line. 



i I • ! t I t 4 »■ a 

0.28 < 0.8 

i decimal as another vay of 
writing a fraction. 



0.17.^ 0.8. A 



. decimal as a part of a region. 



0.38 0.7 

i decimal as an extension of 
place value 




0,17 as 



mum 



i decimal as ^ series. 



i decimal' as a comparison. 



Unit rod 
0.6 
0.«5 



Relate operations with decimals to operations with fractions. 
Ex: 0,7 ^ 0,6 m Q 



But 0.7 ■ aDd 0.6 • 4r 

10 10 

So 0.7 -0.6.ix^ 

U2 
' 100 



Therefore 0.7 x 0.6 ■ 0.U2 



Relate operations with decimals to operations with whole 
numbers, teaching rules for placing the decimal point. 
Ex: 1.38 X 5.2 . Q 

Since 1.38 
X 5.2 
27? 

717? 

1»38 X 5.2 . 7.176 
2 placet 1 plao€ Tptaoma 

Use concrate aaterials to illustrate operations with decimals. 
Ex: 3.U7 ♦ 2.13 - Q 

Using rods or match sticks » I demonstrated that 

3.lt7 ■ 
and 
2.13 m 
makes 
5.60 m 



mmimiim 



wm/mimin 
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1 he acmevement items (more about these later) were divided over several test forms, because there 
were far too many to ask students to answer them all. Each student answered the items in two forms. All 
items, however, were reproduced in the Teacher OTL booklets, where teachers were asked whether the 
students were likely to answer the item correctly and whether they had taught the material (see Section 
2.5). Other forms ontained questions about the schools, about students' and teachers' background and 
attitudes, and all this information was supplemented by an extensive analysis of the curriculum carried 
out by staff at the national centres and experienced mathematics educators. The pool of achievement 
items and the innovative questionnaires were some of the many useful residual benefits counti i.'^^s retained 
after participating in SiMS. 



1.1.1. Curriculum and Teachers 



If the test items were to reflect the mathematics being taught, the first step had to be a curriculum 
survey. Since most countries have a national curriculum, documentation of this national curriculum was 
interesting by itself. Everyone recognized, however, that what was actually taught did not always 
correspond to what was intended. Moreover, students seemed to know some mathenratics that they were 
apparently not ever taught and, more often, to be unaware of topics teachers were sure had been taught. 
Thus was born the overall design concept sketched in Figure 1-3, that there were actually three 
curriculum s everywhere-the official, or intended curriculum, the actually taught, or implemented 
curriculum, and finally what students learned, the achieved curriculum. Chapter 2 ic devoted to the 
curriculum study and teacher survey. 

In almost every respect, Ontario Grade 8 teachers were similar to those in other participating 
countries. Teachers everywhere were experienced (12 to 16 years, except Thailand), their average age was 
36 (40 in Japan) and there were more males than females (except Thailand). The striking exception was 
their degree of specialization in mathematics. In every other country, over 80 percent of mathematics 
teachers specialized in teaching the subject, while in Ontario the figure was 38 percent (either a Type A or 
Type B Mathematics Certificate or an Intermediate Certificate with a mathematics option). Teachers in 
other countries reported spending most of their time teaching mathematics, while Ontario Grade 8 
teachers listed many subjects. This difference was also reflected in the amount of mathematics studied 
during their preservice training-less in Ontario than elsewhere. 

Teachers in most countries said that their mathematics classes were easy to teach. The exceptions 
were Thailand and Japan, where 30 to 40 percent of the teachers reported that mathematics was a difficult 
subject for them. At the extremes, over 80 percent of the teachers in Canada, New Zealand and the USA 
felt that mathematics was easy to teach, but only 8 percent of Japanese teachers felt that way. (The 
Japanese students echoed the opinions of their teachers, reporting m their questionnaires that 
mathematics was more difficult and less enjoyable in comparison with students in other countries.) 

When asked what factors were most responsible for lack of satisfactory progress in learning 
mathematics, teachers were most likely to find fault with the students. Student lack of ability or 
motivation were the causes most often given. Possible causes seldom chosen on the questionnaires 
included misbehaviour, fear of math (chosen by some teachers in France), absenteeism (mentioned in 
British Columbia), and lack of resources (chosen more often by Thai teachers). Poor teaching was chosen 
only by the Japanese teachers as a possible reason why students failed to make satisfactory progress. 

Teachers everywhere have their own view of their subject, what is most important to emphasize-the 
overall objectives. Examples would be, understanding the nature of proof and learning to perform 
computations with speed and accuracy. Where the first is emphasized, it marks a more formal approach. 



Figure 1-3: An expanded model for the three curric;-.(uins 

in the design of the second inter nati ji lal mathematics study 



I. Systems Level 
INTENDED CURRICULUM 



1 



Educational Institutions and Structures I 

! i ' 



Curricular Goals 
and Influences 



Teacher Education 
Values, Practices 



[ 



Textbooks 



m 



Teacher Beliefs 
and Attitudes 



I School and 
Classroom 
Resources 



Teacher Knowledge: 
Methods and Practices 



I Classroom Procygsses 



11. Classroom 
Level 

IMPLEMENTED 
CURRICULUM 



Prior Knowledge 
and Attitudes 



Achievement 
and Attitudes 



III. Student Level 
ATTAINED CURRICULUM 



one that is not recommended for Grade 8 in Ontario. The second represents an objective pursued to some 
exteuc everywhere, so it is instructive to know how much importance teachers give to it relative to other 
objectives. 

Nine objectivos were presented in one questionnaire, and teachers were asked to rate the relative 
emphasis they were giving to each one on a three-point scale. Which nine are selected for inclusion in the 
questionnaire is therefore quite important, since teachers might regaro an objective not listed as more 
important than any of those on the Jist. The relative emphasis they give to each one will be influenced by 
the choice of objectives with which they are to corapare. Teachers do have different views of mathematics, 
since no one of the objectives was consistently emphasized Li all countries. Responses were summarized by 
calculating the percentage of teachers in each country who felt that a given objective should receive more 
emphasis than was accorded to others in the list. 

The nine objectives selected (with percentage of Ontario teachers in parentheses) were: 

1. Understand the logical structure of mathematics. (39%) 

2. Understand the nature of proof. (11%) 

3. Become interested in mathematics. (39%) 

4. Know mathematical facts, principles, and algorithms. (30%) 

5. Develop an attitude of inquiry. (43%) 

6. Develop an awareness of the importance of mathematics in everyday life. (58%) 

7. Perform computations with speed and accuracy. (35%) 

8. Develop an awareness of the importance of mathematics in the bade and applied sciences. 
(19%) 

9. Develop a systematic approach to solving problems. (63%) 

Objective 2 was given low ratings by all countries except France and New Zealand, suggesting that 
teachers in these countries take a more formal view of mathematics at this level. Objective 8 was rated low 
by everyone, indicating, one assumes, that this understanding, however important, will have to come 
later. Objective 6 was favoured by a majority of teachers only in Ontario and the USA. More teachers in 
British Columbia gave high ratings to objective 9 than any other country, and teachers in France were 
very much more in favour of developing an attitude of inquiry (objective 5). 

One would not attempt to draw many conclusions from these responses to abstract statements of 
objectives if they were the only evidence available. In the context of SIMS, however, they are quite 
consistent with other evidence for great diversity among countries in the way the teachers construe the 
content of the mathematics curriculum. Teachers agree on many topics, but how they approach those 
topics, what emphasis they give to them, appears to vary considerably both between and among countries. 

To probe these perceptions in another way, teachers were presented 15 statements about the nature 
of mathematics and asked to respond on a five-point scale, from Strongly Disagree to Strongly Agree. The 
15 items had been shown in pretesting to form a scale that was named Mathematics as a Process. A high 
score signals a teacher who sees mathematics as a field that is changing, growing and developing in new 
directions, while a low score suggests a view of mathematics as static, completely determined and 
unchanging. The items are listed in Table 1-1. 

There were no differences among countries in scores calculated from the 15 items as a whole. There 
were consistent differences in responses to some individual items, however. Very few teachers disagreed 
with statement 5, Mathematics helps one think according to strict rules. On the other hand, there was 
overwhelming agreement with statements 6, 7, and 15 (and disagreement with 8). Overall, teachers 
everywhere gave modest support for a process view rather than a static view of mathematics. Ontario and 



Table 1-1: Items Making Up the Scale, Mathematics as a Process 



1. Mathematics will change rapidly in the near future. (63%) 

2. Mathematics is a good field for creative people. (50%) 

3. There is little place for o/iginality in solving math problems.* (28%) 

4. New discoveries in mathematics are constantly being made. (52%) 

5. Mathematics helps one think according to strict rules.* (76%) 

6. Estimating is an important mathematics skill. (86%) 

7. There are many different ways to solve most math problems. (78%) 

8. Learning mathematics involves mostly memorizing.* (15%) 

9. In mathematics, problems can be solved without using rules. (44%) 

10. Trial and error can often be used to solve a math problem. (75%) 

11. There is always a rule to follow in solving a math problem.* (55%) 

12. There have not been any new discoveries in math for a long time.* (32%) 

13. Mathematics is a set of rules.' v45%) 

14. A mathematics problem can be solved in different ways. (6t'%) 

15. Mathematics helps one to think logically. (94%) 

* =Item marked in reverse order tx> others in calculating scale score. 

The numbers in parentheses are the percentage of Ontario teachers who agreed with each item. 



Thai teachers were more in agreement than the others that mathematics will change rapidly in the near 
future. The Flemish in Belgium held views opposite to just about everyone else, especially the Japanese, 
about the usefulness of trial and error in solving mathematics problems. The Belgian teachers, on the 
Flemish side at least, said it was not useful, while the Japanese teachers said it could be very useful. 

When we come to the ways teachers allocate their time among their various duties, the diversity 
disappears. As can be seen in Table 1-2, there is little variation among countries in the reports teachers 
give of the time they spend preparing lessons, grading papers and the like. They also report about the 
same proportions spent explaining new content, reviewing and testing. They said they spent little time 
discipliiiing and administering and lots of time explaining, reviewing and testing. In another section of the 
questionnaire they also reported that most of their time was used in whole-class instruction rather than 
small-group or individual work. 



1.1.2. Schools and Student!^ 

The first mathematics study had an elaborate plan for choosing participating schools, teachers and 
students with several populations and subpopuUtions. The plan was rarely implemented and drew 
numerous criticisms that the SIMS planners were determined to avoid. Only two populations were 
selected for the second study, (a) the grade in which the modal age is 13, and (b) mathematics specialists, 
students in the last year of secondary education who are taking advanced mathematics as a substantial 
(five hours per week) part of a program leading to post-secondary education. Countries were urged to 
select two classes per school, wherever possible. Strict sampling rules were laid down and supervised by 
an international sampling referee. In spite of these efforts, the resulting samples differed in ways that are 
important to note when attempting any comparisons of results from country to country. 
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Table 1-2: Teacher Reports of Allocation of Teaching Time 
among Their Various Duties 



— ^ 


or Li 






rKA 


JPN 


NZE 


THA 


USA 


Preparation (min/week) 


80 


80 


60 


90 


90 


40 


60 


60 


Grading papers (min/week) 


90 


60 


60 


100 


60 


30 


180 


80 


Explaining (%) 


50 


26 


30 


27 


33 


19 


45 


36 


Reviewing (%) 


25 


17 


25 


23 


22 


12 


20 


20 


Administering (%) 


15 


6 


10 


4 


6 


5 


10 


10 


uiscipiimng yTo) 


2 


4 


5 


2 


6 


5 


8 


4 


Testing (%) 


11 


17 


15 


22 


11 


6 


15 


18 



BFL: Belgium/Flemish; CBC; Canada/British Columbia; CON; Canada/Ontario; 
PRA: Prance; JPN: Japan; NZE: New Zealand; THA: Thailand; USA: United States. 



Virtually every characteristic of school showed considerable variation among countries. Table 1-3 
lists the per school averages of four characteristics for the eight countries in the longitudinal study- 
enrolment, pupil-teacher ratio, length of math period and hours of math per year. Ontario is unusual for 
the relatively short math periods-40 minutes as compared with 50 minutes to one hour in other countries- 
and small schools. Ontario has one of the smallest average enrolments of any country, about the same as 
the Flemish schools in Belgium (but Ontario has a much higher pupil-teacher ratio). The total number of 
hours per year in Ontario equals or exceeds that in all but two other countries. The total number of hours 
per year is smallest in Japan, a surprising result in view of the good achievement there (achievement will 
be discussed in section 4.1). 

Table 1-3: Characteristics of Schools Participating in the 
Population A Longitudinal Study 





BFL 


CBC 


CON 


FRA 


JPN 


NZE 


THA 


USA 


Average enrolment par school 


366 


575 


375 


610 


714 


848 


1293 


548 


Pupil-teacher ratio 


9.4 


17 


19.8 


16.9 


22 


18.8 


28.7 


15.2 


Length of math period 


50 


60 


40 


55 


48 


60 


50 


48 


Hours of math per year 


140 


120 


132 


130 


101 


130 


120 


144 



Recall that Ontario chose Grade 8 for its Population A. This is the grade where most 13-year«olds 
are found, but Grade 8 is also an interesting point for a summative assessment because it is also the 
normal last year of elementary school. British Columbia and the USA also chose Grade 8, but Japan 
sampled classes from Grade 1 of lower secondary school, equivalent to Ontario Grade 7. Other countries 
made modifications to the definition, with the result that the average age of participating studencs varied 
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from 13.2 years to 14.8.^ In May 1982, Ontario's students averaged 13.9 years of age and British 
Columbia's 14.0. 



Students in SIMS were supposed to be a representative, random sample from their population, but 
this was not always achieved. In British Columbia, a careful random sample of schools was obtained, but 
only one class was chosen per school. Wherever there was more than cne Grade 8 class in the school, the 
principal was allowed to choose the class (Robitaille, O'Shea, ijirks, 1982). In Ontario, the SIMS team at 
OISE chose two classes at random from a list supplied by the principal, wherever there was more than one. 
As will be seen, the resulting sample in B.C. was unusual in several respects. In England and Wales a 
number of schools declined to participate, putting the representativeness of their sample in doubt, and the 
same was true in the USA. Similar problems in other countries may have gone undetected. The excellent 
cooperation in Ontario was encouraged by a memorandum from the Ministry of Education to directors of 
boards designating SIMS as an official study. Some details of the Population A samples are given in Table 
1-4 for the eight countries that tested achievement both at the beginning and at the end of the school year. 



Table 1-4: ( jharacterijitics of the Population A Samples 

for the Eight Longitudinal-Classroom Process Countries 



Country 


Number of 


Number of . 


Number of 


Number of 


Strata 


School? 


Classrooms 


Students 


Belgium (Flemish) 


16 


168 


175 


4519 


British Columbia 


6 


90 


93 


2567 


Ontario 


24 


130 


197 


6284 


France 


8 


184 


365 


8778 


Japan 


19 


210 


211 


7785 


New Zealand 


6 


100 


196 


5978 


Thailand 


13 


99 


99 


4030 


U.S.A. 


7 


161 


302 


8372 



Students were asked to name and also to describe their mother's and their father's occupation and to 
tell how much education their parents had. This information was used to derive an indicator of social and 
economic class, since such indicators have always been found to be highly correlated with student 
characteristics, including achievement. In comparison with other countries, Ontario's Population A had 
an unusually large proportion of fathers in the category, semi-skilled and unskilUd workers,^ British 
Columbia had the highest percentage of fathers in the Professional or Managerial category of any of the 
countries in the study, while Ontario was right at the median of that category. Mother's and father's 
education were consistent with the pattern of occupations. 

The definition of Population B as mathematics specialists in the last year of secondary school 
resulted in at least as much diversity among country samples as in Population A. Only a few examples can 
be given here. In most of the world, there is an elite, university-bound group that fits this definition well, 
in Europe, Africa and the Far East, for example. In the USA and British Columbia, however, there are 
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only 12 years of secondary school and the majority of students do not take calculus. Ontario and Scotland 
had both a 12-year and a 13-year system at the time the study was done, and samples of students were 
drawn from both for the study. For the international analyses, Ontario submitted only the sub-sample of 
specialists, whereas Scotland submitted results from all students. The percentage of fathers in B.C.'s 
Population B wheso occupation was classified as Professional or Managerial was above the median, while 
the percentages in Ontario and Scotland were below the median. 

A statistic often used for judging the comparability of Population B samples is retention rate, the 
percentage of eligible students still in school when the sample is taken.^ Because these rates were used to 
help explain the Population B results in the first study, and because they are interesting in their own right, 
retention rates estimated from the second study are reproduced in Table 1-5.^ 



Table 1-5: Percentage of Population B Students in Relevant Age 
or Grade Groups for Each Country: 1981 



Country 


Age Group 
(Years) 


Mathematics 
Retention 
Rate 


Mathematics 
Participation 
Rate 


General 
Retention 
Rate 


Belgium (Flemish) 


17 


9-10 


25-30 


65 


British Columbia 


17 


30 


38 


82 


England & W&ies 


17 


6 


35 


17 


Finland 


18 


15 


38 


59 


Hungary 


17 


50 


100 


50 


Israel 


17 


6 


10 


60 


Japan 


17 


12 


13 


92 


New Zealand 


17 


11 


67 


17 


Ontario 


18 


19 


55 


33 


Scotland 


16 


18 


42 


43 


Sweden 


18 


12 


50 


24 


U.S.A. 


17 


13 


12-15 


82 



Taken from Miller and Linn (1 385). The second column is not always a simple product of the third and fourth columns because not all 
students in the fourth column i /ere in the grades from which the Population B sample was drawn. 



Relative to other countries, a high percentage of Ontario students specialize in mathematics. This is 
seen in Table 1-5 (column 2), where Ontario has the fourth highest percentage of age group in the 
Population B sample, even though the percentage of the age group still in school is lower than most. The 
high percentage of age group still in school in the USA and B.C. (column 4 of Table 1-5) reflects the high 
proportion of students continuing through Grade 12, but note that the percentage in Japan is even higher. 
These data are discussed again in Section 4.2. 1. 
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Unlike the first study, the* finding from the second study was that retention rates had little impact on 
achievement in comparison with other variables (Miller & Linn, 1985). This was so whether one looked at 
the top students (top 1 percent and top 5 percent on SIMS achievement subtests) or at the whole Population 
B (see Section 4.2.3). The conclusion was that the curriculum (particularly opportunity to learn) was the 
strongest factor, but no one factor was adequate to explain the variations in achievement. 

L2c Supportiiig Classroom Instruction 

Students who spend more time on homework get higher marks on tests and do better in school generally. 
This was found in the first mathematics study and has been confirmed in many other studies since then 
(Raphael, Wahlstrom, & Wolfe, 1985). This might be because better students just do more homework or 
because teachers assign more homework to high achievers, but the association is certainly strong and 
widespread. A researcher in the USA pulled together results from many studies and concluded that 
homework can enable less able students to perform up to the level of their more able classmates. Teachers, 
however, tended to put lower demands on the less able students, widening the gap rather than narrowing 
itJ 

In the second mathematics study, students were asked how much mathematics homework they did 
and how much homework they did in all subjects. The highest reports came from Thailand, Belgium, 
Prance and Hungary and the lowest from Sweden, England and Scotland. As was the case so often, Ontario 
was at the median (with the USA) with 2.6 hours of mathematics homework per week, half of the total 
homework done. This total agrees with reports from a large science survey done the next year (McLean, 
1986). B.C. was just below the median. The total amount of homework reported by SIMS participants in 
the USA (5.2 hours per week) was higher than that found in the large study mentioned above (see Note 7). 
That study reported a total figure of four hours per week in the USA, just a little more than the average 
amount of television watched every night Mathematics homework accounts for a large proportion of total 
homework everywhere. In thi ae quarters of the countries, mathematics homework accounted for 35 to 50 
percent of the total, with Nigeria (NGE) and Swaziland standing well out from the others at 79 and 80 
percent. 

Calculators were widely used, but by no means universally so. At the time SIMS was conducted, 
four-function calculators were used by 25 per cent or more of Population A students in 80 percent of the 
countries, either in the schools or at home. Almost 30 p ^ent said that they were used for recreational 
purposes. The highest percentage use was in France and Ontario (67 and 68 percent), and the lowest in 
Nigeria and British Columbia (3 percent). Scientific calculators were used by most Population B students 
(over 90 percent in England and Sweden), with the exception of Hungary (18 percent) and Japan (27 
percent). The low percentage reported by the Japanese appears odd, since they make or sell most of the 
scientific calculators in the world, and their high-achieving students said on their questionnaires that use 
of calculators and computers are necessary to mathematics learning. Computers were not widely used at 
either level, with only New Zealand, Sweden and the USA reporting use by as many as 20 percent of 
Population B students. 

Unlike language, mathematics is not learned at home, from television, from cinema or in the street.^ 
In a 1981 survey (McLean, 1982b, p. 9), only 4 percent of Grade 8 students reported that they received 
regular help with their mathematics lessons from any of their family members, and only 45 percent said 
they received help "once in a while". Girls received occasional help slightly more often than boys. The 
figure drops to 2 percent for advanced level courses in Grade 10. Roughly the same percentages were found 
in SIMS, with the same gender differential. Parents tell children that mathematics is important and that 
it is important to do well, but they do not (perhaps cannot) provide regular help with the mathematics 
lessons. 

Er|c -11. 
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Out-of-school tutoring by persons other than family members is another way of supporting 
clasaroom instruction, and 10 to 20 percent of students at the Population A level receive one to two hours of 
tutonng each week in just about every country. Such tutoring is rare, however, in Sweden ai d in England 
and Wales. There are fc. t countries, where it is quite common. In Japan, Nigeria, Swaziland and 
Thmland, more than half the students receive an average of three hours per week of extra assistance. The 
juku, a private tutoring class, has long been recognized as a feature of Japanese education, but the 
prevalence of such schools in the other three less developed /-ountries was not so widely known. Japan, 
Finland and Hungary also have many such classes at the Population B level. 

The Japanese juku are quite controversial, with almost all educators of the opinion that they 
contribute little or nothing to children's education (T. Sawada, personal communication). Teaching is done 
by parents and retired teachers, few of whom have any knowledge of tl g curriculum. A few juku do offer 
systematic preparation for the state examinations, apparently with some success. Most drill the students 
in traditional calculations and application of standard rules, work the students a (ready get a great deal of 
in their regular classes. 
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Chapter 2 

The Official Curriculums-More Commonality than Diversity 



2.1. Documenting the Intended and Implemented Curriculums 

In most countries, the intended curriculum is easy to document, because it is prescrib.»d by the Ministry or 
the Department of Education. This is the case in Ontario, ^here the Ministry of Education issues 
curriculum guidelines that schools and teachers are obliged to follow. These docuir^ents are just that, 
however, guidelines, and they specify many options at each grade level. Schools are expected to prepare a 
course of study y giving details of their decisions on options, and teachers make decisions from year to year 
and class to class. Diversity is therefore expected, but the extent of variation from school to school and 
class to class surprised most observers. 

This variation was documented in several ways. First, there was a background questionnaire for all 
teachers, asking about their experience and their preparation for teaching in some detail and then asking 
them to report how many periods per week they taught common fractions, decimal fractions, ratio and 
proportion, percent, measurement, geometry and algebra (formuU e and equations). Much more detailed 
reports were provided by teachers on Classroom Process Questionnaires y one questionnaire for each of the 
abovrt topics and one about teaching in general. One page asking about the teaching of the multiplication 
of integers is displayed in Figure 2-1.^ 

2.2. Assembling Items and Matching to the Curriculums 

As soon as a dozen countries agreed to participate, copies of the intended curriculums were obtained and 
international grids were prepared for Population A and Population B. Theae were the large survey 
equivalents of test blueprints-detailed outlines of the content of the curriculums. Mathematics educators 
in each country went over each grid and rated the topics on a scale from very important to not important. 
Combining these ratings, each topic was classified as Very Important, Important, Important in Some 
Systems or Not Important. 

While these ratings were being collected and studied, the task of item writing and assembly was 
begun. Since textbooks are influential in determining what is taught, an analysis of the textbooks in 
common use in each country was made a part of the design and of the process of choosing achievement 
items. In order to make comparisons with the first international study, 35 items were retained from that 
study for SIMS Population A (Grade 8 in Ontario) and 18 fcr SIMS Population B (Grades 12 and 13). All 
items were of the multiple-choice variety, with five alternatives. Items were written as needed to cover all 
topics in the grids, circulated to countries and eventually approved by the International Mathematics 
Committee for inclusion in the study. Almost all were given three trials in eight of the countries as part of 
the selection process. 



Figure 2-1: Questions for teacherr> about the way they taught multiplication 

of integers to the target class. 



9DB8P01I8E CODE 

a. ftiphaslsed (ua*d at a primary method of dercilopBeiit , 
referred to extenairely or frequently). 

b. Used, but not eapbariixed. 
Hot used. 



70. Developaent by use of repeated addition. 

I developed the concept of multiplication 
by appealing to repeated addition, e.g.. 



72. 



3 ♦ '3 ♦ "3 ♦ *3 



12 



71. Deraloj^eiit by the ezteoslon of properties 
of the whole miaber system. 

I d»Teloped the concept of aultlpllcation 
• by using the ccwutatlre, assoclatlre, 
and dletributlre properties to Justify 
fae produ c t s , e.g., 

-Ii K -3 - □ 

0 • 0 X "3 

0 - Ck ♦ %) X "3 

0 • ("H X "3) ♦ Ck X '3) 

0 • X •3) ♦ 'x^ 

Hence {'k « "3) is ths additlre 
inrerse of ~12. 

X '3 • 



Developaent by uae of physical situations. 

I developed the concept of nultlpllcatlon 
of Integers by appealing to phyaicsl 
situations thst sight lllfistrete the 
product of positive and negative nui^ers. 

Ex: A refrigerator Is cooling at a rate 
of 4^/mln. Its thermometer Is 
currently at 0^. What viu be 
its temperature 4 nln from now? 



73. Developaent by use of patterns. 

I developed the concept of multiplication 
of Integers by appealing to patterna of 
products . 

' - . 12 



♦ 



3 ^ 3 ■ 
'*'2 X ~3 ■ 
^1 X "3 ■ 
0 X *3 - 
"l K '3 ■ 
"2 K "3 ■ 



7l». Ko development— students vere given rules. 

I did not develop the concept of 
multiplication of intagers Xjy using any 
of the above methods. Inatiiad, I gave 
the students rules similar to the 

folloving: 

If tbt signa are alike, the anaver 

is positive. 
If the signs are different, the 

aoGver la negative. 
If either factor la aero, the 

answer la tero. 
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Items were prepared first in English and then translated into the language of the participating 
countries. Two independent translations were recommended, and problems with translation were 
reported to the central office of the study. Some items were dropped because no satisfactory version could 
be found for all countries. In the end, an initial pool of 480 icems for Population A and 400 for Population B 
wore winnov^ed to 199 and 136, respectively.^^ The distribution over achievement subsets is shown in 
Tables 24 and 2-2. 

Table 2-1: Structure of the Final Population A Item Pool 
in the Second International Mathematics Study 



POPULATION A 



Achievement 
Subset 



Longitudinal 
Version 



Cross-Sectional 
Version 



Common to 
Both Versions 



Arithmetic 
Algebra 
Geometry 
Measurement 
Descriptive Statistics 

TOTAL 



62 (34%) 
32(18%) 
42(23%) 
26 (14%) 
18(10%) 



180 



46 (2r%) 
40 (23%) 
48 (27%) 
24(14%) 
18(10%) 

176 



46 (29%) 
29(18%) 
40 (25%) 
24(15%) 
18(11%) 

157 



There were 199 items in the final pool. Countries participating in just the end-of-year testing used a cross -sectional 
subset of 176. So did Japan, but the other seven countries that tested at the beginning and end of the year used a 
longUudinal subset of 180 items. A t-tal of 157 items were in both the cross-sectional and longitudinal subsets, and 
these 157 were used in some analyses--the studies of OTL,for example. 



Table 2-2; Structure of the Final Population B Item Pool 
in the Second International Mathematics Study 



POPULATION B 


Sets, relations, functions 

Number systems 

Algebra 

Geometry 

Analysis 

Probability and Statistics 
Not in subtests 


7 (5%) 
17 (13%) 
26(19%) 
26 (19%) 
46(34%) 
7 (5%) 
7 (5%) 


TOTAL 


136 



3 combinatorics items, 3 low coverage, 1 found flawed 
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How well the SIMS item pool matched a system's intended curriculum was measured by calculating 
the percentage of items in each topic subset that educators said were either highly appropriate or 
acceptable to that system. In other words, the index oUoverage represents the proportion of items in the 
SIMS pool that are based on mathematics intended to be covered, as reported by experienced mathematics 
educators. 



2.3. Intended Curriculum Coni;ont: Grade 8 

Figure 2-2 summarizes the indices of intended Population A content coverage for those educational 
systems for which appropriateness ratings were available. Ontario intends to teach a higher percentage of 
the SIMS items in arithmetic, geometry, statistics and measurement but not so many in algebra. By 
inspection of the table it can be seen that education systems are most alike for arithmetic and 
measurement and most different for geometry and statistics. Given that the international intended 
curriculum was perceived as adequate to cover most mathematics programs, Ontario's overall high rank 
on intended coverage (sixth among countries) reinforces the relevance of the SIMS item pool to Ontario 
schools and encourages continued amalysis. 

Figure 2 2: Percentage of SIMS Population A items countries 
intend to cover, by topic, for each country. 



Percent 
ofltenu 
Covered 



Aritliuietic 



Algebra 



Geometry 



Statistics 



Measurement 



too 

90 

80 
70 



CON SCO USA 



BFL BFR HKO 
ISR HUN IRE 
LUX NZE JPN 
THA CBC ENW 

FIN FRA SWA 
SWE NTH 



IRE NZE 



FRA HUN SCO 
BFL BFR JPN 



CBC ENW NTH 



ISR THA SWE 
LUX FIN HKO 
SWA 



JPN NZE SCO 



ENW CON HUN 



FIN ISR THA 
NTH 



NZE SCO SWE 
USA 



JBC 



ENW FIN HKO 
HUN CON SWA 



JPN THA 



CON ENW HUN 
ISR JPN NZE 
SCO THA USA 

FIN HKO IRE 
CBC SWE 



NTH SWA 



LUX 



USA CON 



50 
40 
30 
20 
10 
0 



SWA HKO 
SWE USA 
CBC FRA 
IRE 

BFL BFR LUX 



NTH IRE 



FRA BFL BFR 



LUX FRA 
BFL BFR 
ISR 



Natems 62 42 51 18 26 

Avg.Pct. 92 83 64 69 91 
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Having the highest intended coverage for arithmetic may not be the most desirable outcome, 
however. As will be discussed in Chapter 4, it might be more appropriate to transfer part of this effort to 
algebra. The issue with geometry is different. Although the SIMS items were rated highly appropriate, 
many teachers simply did not get around to teaching geometry or gave it low priority for instructional 
time. Teachers rated fewer algebra items as appropriate or acceptable, because the topic is seen as one 
more suited to the secondary school. The clear exception was integers, included under algebra in SIMS and 
very widely taught in Grade 8. 

2.4. Intended Curriculum Content- Grade 13 Specialists 

Provincial data were gathered and reported in the Ontario national report (McLean, Raphael, & 
Wahlstrom, 1986) for Grade 12 and for all three Grade 13 courses, but because of the narrow definition of 
mathematics specialists (students taking two or three of the Grade 13 courses) the Population B intended 
coverage was judged only against the Grade 13 courses. The summary of intended curriculum coverage is 
presented in Figure 2-3. 

Figure 2-3: Percentage of SIMS Population B items countries 
intend to cover, by topic, for each country. 



Percent 
of Items 
Covered 



100 



90 



80 
70 
60 

50 
40 
30 
20 
10 
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Seta, Rel 
and Func 



BFL BFRCBC 
CONPINFRA 
HKOISRLUX 
NZESCO 



IREJPNTHA 
USA 

ENWHUNSWE 



Algebra 



BFLBFRCON 
ENWHKOIRE 
NZESCOTHA 



FRAHUNJPN 
LUX SWE FIN 
ISR 



CBC USA 



Geometry 



Elementary 
Function 
and Calc. 



Prob. and 
Stat. 



HKO 



CONJPN IRE 
FRALUX 
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NZE 
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USA 

CBC THA 



NZE 



FRAIRE LUX 
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FIN HUN ENW 
HKO ISRJPN 
USA 
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THA 



CBC 



CON ENW FIN 
FRA IREJPN 
NZE 



HKO ISR LUX 
SWE THA USA 



Finite 
Math 



CON FIN FRA 
HKO IRE JPN 
LUX NZE SCO 
THA 



BFL BFR ENW 
SWE 



USA 



SCO 

BFL BFR HUN 
CBC 



CBC 



HUN l?K 



N. Items 
Avg. Pet. 



7 
92 



25 
96 



28 
73 



46 
89 



7 
76 



4 

76 



-17- 



Eighty percent or more of the mathematics necessary to answer the algebra items was intend 
taught in all educational systems, whereas many more of the geometry items were not thougF 
appropriate or acceptable. Ontario and Japan rated 80 percent of the geometry items as appri 
whereas British Columbia and Thailand reported only 50 percent of them as appropriate. The two i 
elementary functions and calculus and geometry were least well covered in Ontario in comparison i 
educational systems, but the coverage was still 80 percent British Columbia and the USA often 
low coverage because their systems only went to Grade 12. Not surprisingly, there was the ^ 
variation in coverage of finite mathematics, a topic that has still not achieved high status ii 
countries in spite of its many applications in fields such as computing. 

2.5. Implemented Curriculum Content 

One of the important objects of study in all recent lEA projects has been the variable descr 
"opportunity-to-leam" (OTL). The SIMS design asked classroom teachers to complete the "1 
Opportunity-to-Learn Questionnaire" in which they were asked the following questions about each 
the student test forms used in the classroom: 

1. What percentage of the students from the target class do you estimate will get the item corre< 
without guessing? 

• Virtually none 

• 6-40 percent 

• 41-60 percent 

• 61-90 percent 

• Virtually all 

2. During this school year, did you teach or review the mathematics needed to answer the itei 
correctly? 

• yes 

• no 

3. If, in this year, you did NOT teach or review the mathematics needed to answer this itei 
correctly, was it because, 

• It has been taught prior to this school year? 

• It will be taught later (this year or later)? 

• It is not on the school curriculum at all? 

• For other reasons. 

Students were also asked to answer for each item whether they had been taught the material 
year, (b) before this year or (c) not at all. The index derived from these responses is called student 
distinguish it from the one derived from teacher questionnaires, teacher OTL, The OTL variabl 
included to measure the extent of the implemented curriculum, but as of this writing only the teach 
data have been analyzed. Ontario administered the Teacher Opportunity-to-Learn Questionnaire : 
grade levels, one of 16 systems to do so for Population A and one of 12 systems for Population B. 

Teacher OTL taps not only the curriculum of the target year but also the background knowlec 
teachers believe students bring to the testing situation. A difficulty with this measure is that t 
sometimes do not know what has been taught in earlier grades, and considerable difference of 
emerged when referring to the three parallel Grade 13 courses. Teachers were asked to choose one 
the target class and to answer all questions with reference to that class, but this may not always ha 
done. (The problem of rating OTL when there were simultaneous courses was also severe in New ; 
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and in England and Wales.) In subsequent studies at the school board level in Ontario, the OTL 
questionnaires were completed by the mathematics department as a unit rather than by individual 
teachers. This procedure assur id information that reflected OTL for the school, and the school was used as 
the unit of analysis, but there is still the problem of knowing which students had what opportunity to learn. 

Figure 2-4 displays the percentages of SIMS items reported by teachers as covered in each country, 
for five topic areas. Canada/Ontario (CON), is moderately above the mean in all areas, the lowest 
percentage (50 percent) being reported in geometry. Thus, the pattern observed in the measures of 
intended curriculum are repeated in the implemented curriculum. This is encouraging, because the 
indices of intended curriculum were obtained from a few people rating the whole system, whereas the 
indices of implemented curriculum are averages over many teachers. For geometry, the primary variable 
influencing the amount of instruction appears to be cultural. Countries whose curriculum is modeled on 
the traditional English curriculum together with Hungary, Japan, the Netherlands and Thailand give 
geometry moderate or substantial emphasis and treat a more or less common core of subjects in plane 
figures and coordinates. The remaining countries-Ontario, British Columbia and the United States- 
devote little attention to the overall area and the topics covered are scattered. 

Figure 2-4: Percentage of SIMS Population A items covered, 

by topic, foi each country. 
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ofltems Arithmetic Algebra Geometry Statistics Measurement 
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ENW THA 

ISR FRA USA 

BFL LUX SWE 
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CON NZE NGE 
ENW 

FIN FRA ISR 
THA 

SWE CBC 
NTH LUX BFL 



N. Items 
Avg.Pct. 



24 
73 



46 

73 



30 
67 



39 
45 



18 
51 




Percentages of Population B items reported by teachers as covered are displayed in Figure 2-5. 
Ontario teachers cover about 80 percent of the algebra items in the SIMS pool. Ontario Grade 13 algebra 
appears to be unique in the topics covered when contrasted with other pre-oniversity algebra courses. In 
th^ case of Ontario's calculus, figures not shown here reveal that there is variability of coverage but that 
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about one half of the classes cover most of the SIMS domain. In Israel and in England and Wales, 
comparatively few teachers cover sets, relations and functions. In Ontario, British Columbia and Sweden 
there are core groups of classes reporting a fall coverage of functions and highly variable coverage outside 
this core. The topic that receives the greatest coverage is algebra. For two topics there was a particularly 
large discrepancy between intended coverage (Figure 2-3) and implemented coverage (Figure 2-5). All of 
the SIMS items were intended for finite mathematics and probability and statistics, but teachers reported 
covering only 30 per cent. 

Figure 2-5: Percentage of SIMS Population B items covered in each country, 

by topic, for each country. 



Percent 
of Items 
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N.items 7 25 28 46 7 4 

Avg.Pct 71 87 62 78 59 61 



2.6. Intended and Implemented 

The curriculum analyns of SIMS is useful for an analysis of the mathematics curriculum, with or without 
the benefit of student achievement data. Boards, or even mathematics departments, can use the 
information to verify that they are teaching the topics they want and giving them the emphasis they want. 
Introduction of the three curriculums helps get away from the emotional discussion of the curriculum 
usually found in basic measurement arid evaluation or curriculum foundation texts. The intended 
curriculum is defined by the official syllabus, but it is augmented by textbooks, course descriptions, 
examinations, and resource documents produced by the Ministry of Education. Boards and schools 
translate the guideline into a course of study, and all these documents combine in mysterious ways to 
define the intended currifiulum. 

Classroom teachers then translate the intended into the implemented. In Ontario the implemented 
curriculum tends to reflect the intended curriculum, because Ontario educators are prolific authors of 
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textbooks that hew to the curriculum docuraented in the official guideline. In the past, it was not 
uncommon to find members of the curriculum guideline writing team (or their close colleagues) 
simultaneously writing the guideline and a textbook. Whereas Ontario is large enough to have its own 
textbook industry, British Columbia must import texts written for different guidelines. Quebec and the 
French-speaking communities outside Quebec usually rely on materials from France and Belgium which 
are then augmented by local distributors. 

During the 1960s the curriculum reform movement was especially active in the areas of physics, 
chemistry, biology, and mathematics with the new mathematics being introduced. This was the time of the 
first lEA mathematics study which only partially captured the innovative spirit of the 60s. The 70s saw a 
stabilization period in which SIMS was designed and vigorous curriculum reforms undertaken in many 
educational systems. From the 1960s to the 70s Ontario changed its approach from writing a detailed 
syllabus to writing curriculum guidelines from which each of the more than one hundred school boards was 
required to detail a mathematics program. Local autonomy was granted in the specific treatment of 
instruction within the classrooms. As well, provincial examinations were eliminated as inconsistent with 
this diverse approach to instruction. 

Thus, more so than in the past, when each school board in Ontario was expected to have the same 
implemented curriculum, it is advantageous to have the SIMS curriculum grid to compare the intended 
curriculum of the province with the curriculum implemented by. local school boards. The diversity meant 
that the detailed analysis of the various curriculums and the settings were even Tnore important. In 
Ontario, 50 schools were selected in which both students and teachers were asked whether the 
mathematics necessary to answer the achievement items had been taught (see Section 2.5), and the 
resulting OTL indices have been compared with the indices of intended curriculum derived from 
appropriateness ratings. 

The difference between indices of intended and implemented content coverage are presented in 
Table 2-3 for Grade 8. For Ontario the weighted total is near zero, that is, the Population A items not 
covered are balanced by topics not in the pool. Geometry and statistics are apparently implemented less 
than policy intends. A typical explanation is that those preparing curriculum guidelines are unduly 
optimistic about what teachers are able to cover during an academic year. In Grade 8, geometry is 
frequently left until the end of the year, when instructional time grows short. It is common knowledge that 
geometry is regularly neglected by implicit design of the instructional schedule. 

Although achievement is discussed more thoroughly in another section, we note in Table 2-4 that 
Ontario teachers are consistently overoptimistic about the performance of their students. This may stem 
from the same optimism that results in greater intended than implemented curriculum. The opposite is 
true in Japan, where teachers underestimated student performance, especially for geometry and statistics. 
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Table 2-3: Index of the Amount by which the Intended Curriculum Is Greater 
than the Implemented Curriculum for Population A Topics 



ERIC 



Country 


Arithmetic 


Algebra 


Geometry 


Statistics 


Measurement 


Weighted 
Total 


r>elgium(i< iemish) 


9 


22 


-6 


-27 


-16 


0 




9 


2 


-7 


42 


18 


9 


oanadayunt. 


-2 


-14 


27 


16 


-6 


4 


England 


19 


30 


33 


19 


18 


24 


r iniana 


10 


8 


30 


33 


23 


.3 


r ranee 


-4 


11 


11 


-23 


-30 


-4 


Hungary 


2 


-1 


-1 


-2 


3 


0 


Israel 


25 


-3 


33 


-47 


41 


16 


Japan 


9 


10 


36 


25 


5 


17 


Luxembourg 


14 


21 


-12 


-10 


-3 


4 


iNetneriands 


8 


7 


11 


29 


5 


11 


ew^/eaianQ 


26 


34 


28 


40 


30 


30 


Swaziland 


2 


-4 


-16 


6 


-5 


-4 


Sweden 


20 


26 


13 


53 


28 


24 


Thailand 


7 


-10 


14 


21 


14 


8 


U.S.A. 


16 


-S 


12 


28 


25 


14 
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Table 2-4: 



Amount by which Teachers' Predictions of Student Achievement 
Exceeded Actual Student Achievement. 



Country 


Arithmetic 


Algebra 


Geometry 


Statistics 


Measurement 


Weighted 
Total 




in* 

-10 


-2 


8 


1 


0 


-1 


A M M /^l rcy 

C/anaua/r>U 


-4 


4 


13 


•7 


2 


2 


uanada/unt. 


2 


14 


14 


0 


3 


7 


England 


-2 


-3 


-10 


-16 


1 


-5 


r inland 


0 


0 


-15 


-14 


-2 


-6 


France 


-5 


1 


14 


2 


-1 


2 


Hunga; / 


3 


4 


1 


-3 


8 


3 


Israel 


3 


-2 


-16 


-25 


-3 


-7 


Japan 


-2 


-1 


-13 


-11 


4 


-5 


Luxembourg 


4 


2 


-5 


-6 


8 


1 


JN etneriands 


-2 


1 


-3 


-18 


2 


-3 


NewZealand 


-6 


7 


-2 


-15 


1 


-2 


Nigeria 


9 


14 


13 


4 


15 


11 


Swaziland 


11 


14 


8 


6 


12 


10 


Sweden 


-6 


-3 


-9 


-15 


-11 


-8 


Thailand 


6 


10 


7 


2 


-2 


6 


U,S,A. 


-4 


5 


11 


-11 


6 


2 



Negative entries appear where student achievement (percent) was higher than predicted by the teachers. 
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Chapter 3 
Attitudes and Achievement in Ontario 



In this chapter we present a selection of student attitude and achievement results from the comprehensive 
report submitted to the Ministry of Education on the study in Ontario (see Appendix B). This will give 
readers the flavour of the detailed information provided by the SIMS instruments. The attitude 
questionnaires covered the nature of mathematics, the place of mathematics in society, students' personal 
feelings toward the subject and their views about calculators and computers. 

These are results from students, but since, in the case of the achievement items, each student 
answered only those in two of the booklets, all results are averages over classes or schools. Where directly 
relevant, teacher reports are compared to the student responses. All the attitude and achievement results 
m this chapter are calculated from averages of responses to individual items over all the students in a class 
or a school. Class or school means are then averaged to arrive at a score for a topic-arithmetic, for 
example, or geometry. 

3.1. A Close Look at Responses from Ontario's Population A Students 

We will first look at achievement, focusing on fractions and on algebra. These topics were chosen because 
there are international results thatsuggekt further consideration of the Ontario guideline in these areas 
The international results are discussed in Chapter 4. Student responses to the attitude questionnaires will 
follow the achievement section. 



3.1.1. Teaching and Learning About Fractions and Algebra 

Nearly all teachers spend time in Grade 8 on fractions, common fractions such as 2/5ths and 3/8ths 
and decimal fractions such as 0.40 times 6.38. They also teach the equivalence of common and decimal 
fractions, such as "7 3/20 is equal to ?". Table 3-1 gives the detailed results for the 12 items making up the 
SIMS subset on common fractions. At the bottom of the table, the summary ^hows that the overall average 
number correct (Right) was 51 percent on the Pretest and 57 percent on the Posttest. Very few students 
failed to answer (Omit), so the modest gain is due to more students giving the correct answer. Looking 
under the column in the middle of the table marked Change, we see that the greatest improvement (15 
percent) came on the item asking for division of fractions, a topic taught by 91.5 percent of the teachers 
The first few words of the stem of the item (sometimes all of the stem) are given at the bottom of the table' 
One quarter of the teachers spent three class periods on this topic (not shown in this table), and only 7 4 
percent of the teachers said that this had been taught prior to Grade 8! 
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Table 3-1: Summary of Ontario Student Responses (percent) and Teacher 
OTL Reports (percent) on the Twelve Items Making Up the 
Common Fractions Subset within the Arithmetic Topic for Population A 



Item* 


PRETEST 
Rght Wrng Omit 


POSTTEST 
Rght Wrng Omit 


Change 


Gr.8 


i augn t 
Prior 


1. 


27 


71 


2 


32 


67 


1 


A 
fr 




o.Z 


2. 


48 


46 


5 


58 


39 


3 


in 




lie 


3. 


57 


42 


1 


63 


36 


1 


e 
o 


Q1 R 


1,6 


4. 


27 


63 


10 


32 


62 


6 


e 
o 


09 O 


iO.D 


5. 


64 


35 


1 


68 


31 


1 


A 
t 


oo.o 


O.O 


6. 


61 


38 


1 


65 


35 


1 


A 
% 


Q9 A 

9^.D 


0.0 


7. 


49 


48 


3 


64 


34 


2 




9i..i> 


1.4 


8. 


51 


46 


3 


54 


44 


2 


A 
% 


9U.U 


0.0 


9. 


81 


17 


1 


85 


14 


1 




QA 9 


O.O 


10. 


49 


48 


3 


56 


41 


2 


8 


92.6 


7.4 


11. 


46 


tiA 
04 


0 


48 


52 


0 


2 


85.9 


14.1 


12. 


55 


44 


1 


61 


39 


1 


5 


91.9 


8.1 


Avg. 
3td. Err. 


51 
1.0 


46 
1.0 


3 

0.3 


57 
1.0 


41 
1.0 


2 

0.2 


6 

0.8 







*First Part of the Item Stem 

1. Four 1-litre bowls of ice cream w^;re set out at a party . . . 

2. In the figure the little squares are all the same size and 
3.2/5 + 3/8 is equal to... 

4. Which of the points A, B, C, D, E on this number line . . . 

5. Which is the closest estimate for the answer to 5-3/7 + 6-5/9? 
6. 3/8 -1/5 is equal to... 

7. (3/5)/ (2/7) is equal to... 

8. There are 35 students in a class. 1/5 of them come to school . . . 

9. Which of the following is a pair of equivalent fractions? . . 
10. 1-2/5- 1/2 is equal to... 

11. The picture shows some black and some white marbles. Of all 
12. 1/2+ 1/4 is equal to... 
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Table 3-2 is exactly the same summary for the 13 items of the decimal fractions subse ,. Here again, 
almost all teachers cover the mathematics required to answer the items (with one or two exceptions) and 
few report that the material was covered prior to Grade 8. The very modest gains (from 47 to 52 percent 
overall) suggest some stubborn learning problems. The greatest gain was on the item mentioned earlier- 
equivalence of a common fraction to a decimal. An 1 1 percent gain was registered on the word problem: 

Alexandra walked from River view to Bridgeport, which are 3.1 km apart. During her walk she 
lost her watch, went back 1.7 km to find it, and then continued in the original direction until she 
reached Bridgeport. How many kilometres had Alexandra walked altogether when she arrived 
at Bridgeport? 

A. 1.4 

B. 4.8 

C. 6.5 

D. 8.2 

E. None of these. 

Teachers strongly agreed that emphasis should be put on teaching applications of fractions. The 
first item on the list in Table 3-1 would appear to fall in this category. Nearly 90 percent of the teachers 
said they taught the mathematics, but the student success rate was near the chance level. 

Fom 1 L bowls of ice cream were set out at a party. After the party, 1 bowl was empty, 2 were 
half full, and 1 was three quarters full. How many litres of ice cream had been EATEN? 

A. 3|L 

B. 2^L 

C. 2iL 

D. 1|L 

E. None of these 

Students clearly knew how to detect equivalent common fractions, since over 80 percent got the item 
(the ninth in the list) correct on both the pretest and the posttest. Nevertheless^ 95 percent of the teachers 
reviewed this topic, the most commonly reviewed among the common fractions topics. 

Algebra topics provide a sharp contrast to fractions. AlgGbra is more abstract, less encountered 
outside of school and in most cases new to the students in Grade 8. An especially clear example is that of 
the integers, positive and negative whole numbers such as -4, 6, 0, -11 and the like. Operations with 
integers (addition, subtraction, multiplication and division) are taught as new content by over two-thirds 
of the teachers, and where not new (addition and subtraction, primarily), they are reviewed. The summary 
of student responses and teacher OTL reports for the five items in the integer subset is presented in Table 
3-3. 

Overall Change of 22 percent refiected the extensive teaching of this new material. The high 
percentage on both pretest and posttest for the last item in the list would appear to reflect successful 
learning of order of operations and the distributive law in prior grades, augmented by practice in 
multiplication. The percentage correct would almost certainly have fallen if the integers in the 
parenthesis had been reversed to (-4 -f 6), but the gain might well have been greater. 
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Item* 



X auit; o-^: nummary oi umano ocuaem Responses (percent) and Teacher 
OTL Reports (percent) on the Thirteen Items Making Up the 
Decimal Fractions Subset within the Arithmetic Topic for Population A 



PRETEST 
Rght Wrng Omit 



POSTTEST 
Rght Wrng Omit 



Chango 



Taught Taught 
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oz 
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14 


96.3 


2.1 
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59 


AO 
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DO 


04 
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63.0 


13.2 


O. 








CO 
DO 


00 
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92.1 


7.9 


7 




01 
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on 


CO 

d3 
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2 


90.5 


9.0 


Q 
O. 




Aft 

Oo 


1 
X 


AO 




1 


11 




lO.O 


Q 








A 1 

41 


57 


1 


2 


66.3 


H. 7 


10. 


45 


49 


6 


51 


45 


4 


6 


72.5 




11. 


53 


46 


1 


62 


37 


1 


9 


95.8 


4.2 


12. 


19 


75 


6 


22 


74 


4 


4 


79.0 


7.0 


13. 


68 


30 


2 


66 


32 


2 


2 


88.0 


10.3 


Avg. 


47 


51 


3 


52 


46 


2 


6 






Std.Err. 


1.0 


0.9 


0.2 


1.1 


1.0 


0.2 


0.7 







First Part of the Item Stem 



1. The value of 0.2131 + 0.02958 is approximately . . . 

2. In a discus-throwing competition, the winning throw was 61.6 . . . 
3.0.40 + 6.38 is equal to... 

4. 7-3/20 is equal to... 

5. The position on the scale indicated by the arrow is . . . 

6. .004)2TSr 

In the division above, the correct answer is . . . 

7. Which of the following is thirty-seven thousandths? . . . 

8. Alexandra walked from Riverview to Bridgeport, which are 3.1 km . 

9. The large square has area 1 square unit. The area of the . . . 

10. The speed of sound is approximately 340 metres per second 

11. 74.236 rounded to the nearest hundredth is . . . 

12. A runner ran 3,000 metres in exactly 8 minutes. What was . . . 



13. 



847.36 In the number in the box, the digit 6 represents . 
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Table 3-3: Summary of Ontario Student Responses (percent) and Teacher 
OTL Reports (percent) on the Five Items Making Up the 
Integers Subset within the Arithmetic Topic for Population A 

PRETEST POSTTEST Taught Taught 

Item Rght Wrng Omit Rght Wrng Omit Change Gr. 8 Prior 



1. 


14 


83 


3 


61 


38 


1 


47 


89.4 


6.4 


2. 


40 


58 


2 


50 


49 


1 


10 


92.6 


2.1 


3. 


16 


81 


3 


43 


55 


1 


27 


94.0 


0.0 


A 

4. 


37 


55 


8 


57 


42 


2 


20 


89,7 


3.2 


5. 


58 


31 


11 


65 


31 


3 


7 


89.1 


0.5 


Avg. 


33 


61 


5 


55 


43 


2 


22 






Std.Err. 


1.2 


1.1 


0.5 


1.4 


1.3 


0.3 


1.4 







*First Part of the Item Stem 
(-2) + (-3) is equal to . . . 

The air temperature at the foot of a mountain is 31 degrees 

(-6) - (-8) is equal to . . . 

The set of integers less than 5 is represented on one of the . . . 
-5(6 - 4) is equal to . . . 



Little formal teaching of algebraic formulas is undertaken in Ontario Grade 8, in contrast to many 
other countries (see Section 4.1.3). Students either failed to respond (9 percent) or guessed on the following 
item (% choosing each alternative given in parentheses): 

Soda costs a cents for each bottle, including the deposit , but there is a refund of 6 cents on each 
empty bottle. How much will Henry have to pay for x bottles if he brings back empties? 

A. ax -f by cents (12%) 
E. ax-by cf^nts (25%) 

C. fa -6;x cents (14%) 

D. (a'hx)-(b'h y) cents (16%) 

E. None of these (24%) 

3.1.2. Grade 8 Students* Attitudes to Mathematics 

In Section 1.1.1, teacher responses to the scale Mathematics as a Process were discussed (see also 
Table 1-1 on page 7). Students responded to the same items, agreeing with the teachers on about half of the 
items. In their disagreements, the students revealed a view of mathematics as slightly more static and 
rule-driven than the teachers. Both agreed that mathematics will change rapidly in the near future and 
that it is a good field for creative people, but the students were less likely to see a place for originality in 



1. 

2. 
3. 
4. 
5. 
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solving mathematics problems. More students than teachers agreed with the statement, Learning 
mathematics involves mostly memorizing, and tended to disagree with the statement, In mathematics, 
problems can be solved without using rules. The most extreme views students had were strong agreement 
with There is always a rule to follow in solving a mathematics problem, and Mathematics helps one to think 
logically. 

The scale Mathematics and Myself v/as presented to students in Grades 7 to 10 as part of the OAIP 
field trials in mathematics and English at the end of the 1980-81 school year and then again in 1982 in 
SIMS. The results were entirely consistent-students believe mathematics is important, they really want 
to do well and their parents want them to do well. They are uncertain about their own abilities, however, 
and do not express much enthusiasm for the subject (McLean, 1982b). 

The eight items of the scale Mathematics in Society are all concerned with the uses and importance 
of mathematics in work and everyday life. To every item, students gave responses that restated their view 
of the importance and usefulness of the subject. A number of students said, however, that they could get 
along well without using mathematics and that it was not needed in everyday life. This is probably a 
genuine, if unfortunate, view of a minority of young people. For them, the mathematics studied in school 
appears irrelevant, and more attention to applications would not be misplaced. One hopes they stay in 
school and find the classes that are relevant to their needs. 

Few students had access to computers in 1982, perhaps the last year that would have been true. 
Tkay were generally of the opinion that "everyone" should learn something about computers, but they 
were undecided whether they personally wanted to learn much and many felt that they could never learn 
to program a computer. The survey should be repeated in a few years. Calculators were widely used, more 
widely than in most other countries, but students were undecided whether calculators helptd them to 
learn. They disagreed strongly, however, with If you use a hand calculator, you do not have to learn how to 
compute. 

We turn now to the responses from the mathematics specialists-from the general population at the 
end of elementary school to an elite sample in their last pre-university year. 

3.2. Another Close Look-Population B in Ontario 

As described in earlier chapters, the Ontario Population B sample was very complex. Four classes were 
sampled in each school, where possible, from the Gride 12 course leading to Grade 13, and one each of the 
three pre-university Grade 13 courses. A smaller proportion of women are usually found at this level, and 
this was true in Ontario-more so in algebra than in other courses. Parents' occupations revealed a 
disproportionate number of students from higher income groups, though as noted in Section 1.1.2, this was 
not as marked in Ontario as in other systems (B.C., for example). We turn first to the achievement results 
and then briefly to attitudes. 

3.2.1. Achievement in Calculus: and Algebra 

As with the Population A discussion, two topic areas are presented that seem to have implications 
for Ontario in the light of international results. The implications will be discussed in Chapter 4, especially 
Section 4.2.2. 

Calculus is a popular subject in Grade 13. About 62 percent of men and 44 percent of women take it. 
It is a traditional course, concentrating on limits of functions, differentiation, definite integrals and the 
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calculus of functions. Analytic integration techniques (trigonometry substitution, integration by 
parts, . . .) are covered in about half the classes, but infinite series and partial derivatives are rarely 
covered. Two SIMS achievement subsets were particularly relevant to Ontario's curriculum, 
differentiation and integration. Summaries of student achievement will be found in Tables 3-4 and 3-5. 

With one or two exceptions in each subset, teachers reported overwhelmingly that they taugh*; the 
mathematics necessary to answer the questions, and achievement was related to OTL. Ontario students 
did reasonably, but not outstandingly, well. Only 28 percent, not many more than by chance, chose the 
correct answer to a standard application item-finding the maximum of a function (item 8, Table 3-4). 
They also had to produce the function, probably the biggest stumbling block. Evidence for an emphasis on 
rules for differentiation is seen in item 7, where 67 percent were able to choose the correct derivative of 
expixh 

Performance on the integration items was more variable, even discounting the three items taught by 
fewer than 80 percent of the teachers. Integrals of standard fimctions were answered correctly by nearly 
80 percent of the students, and 56 percent were able to find the value of a definite integral of x - x*^ (item 9, 
Table 3-5). Only 3^ percent responded correctly, however, when asked to find "the area enclosed between 
the curve y = - x^ and the x-axis" (item 4). Students were completely baffled by item 8, "Given that 3 
f • (x) = x^ - 5, and f(2) = 1, then f(0) is equal to . . a straightforward application of the meaning of the 
derivative and simple integration. 

Ontario students did relatively well on the algebra questions. A summary of results is given in 
Table 3-6. The overaU success rate of 51 percent on these difficult items is good achievement. There is no 
information on teacher OTL because the material might be taught in any one of the three classes and 
students were taking various combinations of them. The average was dragged down substantially by the 
almost imiversal mistake in item 4, 

P is a poljoiomial in x of degree m, and Q is a polynomial in x of degree n, 
with n < m. The degree of polynomial (P -f Q)(P - Q) is . . . 

Most students chose mn, whereas the correct answer is 2m, Performance was good overall, however, which 
mathematics educators attributed to the high proportion of students who were taking the very demanding 
Algebra course in Grade 13. 



3.2.2. Mathematics Specialists' Attitudes to the Subject 

The attitude results can be summarized quickly. The elite sample had a view of mathematics as a 
dynamic, changing field, a good field for creative people. They saw it as important and useful. In contrast 
to the general population in Grade 8, they were more confident and felt good about their abilities. Almost 
none said they could get along well without using mathematics, but a few did allow that a knowledge of 
mathematics was not necessary in most occupations. The interesting finding was a consistent trend 
"•^ward even more positive attitudes as the group became more specialized. These young adults have made 
their choices already, and their replies were consistent with their choices. 



3.2.3. Participation by Girls 

The proportion of girls in school varied somewhat from country to country. One way to compare 
participation by girls was therefore to calculate the change from Population A to Population B in the 
proportion of girls enrolled in mathematics classes. The general experience is that fewer girls take 
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Table 3-4: Summary of Ontario Student Responses (percent) and Teacher 
OTL Reports (percent) on the Fourteen Items Making Up the 
Differentiation Subset within Analysis for Population B 

Response Rates Teacher 



Item 


Right 


Wrong Omit 


OTL 


First Part of the Item Stem 


1. 


83 


15 


2 


97 


A function /"is defined by /(x) = (3a; + 1)^. The derivative 
of /"at X is? 


2. 


55 


41 


3 


97 


The graph of a function /"has a point of inflection at (a,l). 
Which of the following MUST be true? 


3. 


18 


66 


16 


92 


The graph of the equation y = 3x^ + Qx^ + kx + 9is . . . 


4. 


66 


31 


3 


97 


The derivative with respect to x of 4// 3a; - 4 is . . . 


5. 


67 


31 


2 


95 


The velocity of a body moving in a straight line . . . 


6. 


47 


49 


4 


96 


At which point does the curve y = Sa;^ - x^ have a local 
minimum? 


7. 


67 


25 


7 


83 


The function fis defined by f'{x) = e^, fix) = ? 


8. 


28 


59 


12 


88 


The intersection of a cylinder with a plane through its 
aa;is is . . . 


9. 


38 


56 


6 


92 


The function /"defined by /(a;) = x'^ + 4x^ has a relative 
maximum of at . . . 


10. 


51 


31 


18 


94 


The curve defined by ;y = a;^ - ax + 6 has a relative 
minimum pwint at . . . 


11. 


43 


45 




54 


If X - 2 cos t andy = sin t, find dy/dx in terms off. 


12. 


38 


48 


14 


95 


TV iAXiriii ui. bile xuijiuwinig giapno nas tneo6 xeacures . . . ; 


13. 


17 


64 


20 


39 


fis an even function and is differentiable at 0. What 
oondition must/(0) satisfy? 


14. 


23 


45 r 


. 32 


9 


In the a£Fine Euclidean plane, the coordinates of a 



moving m . . . 



Avg. 46 43 11 
Std.Err. 1.3 1.0 1.1 
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Table 3-5: Summary of Ontario Student Responses (percent) and Teacher 
OTL Reports (percent) on the Twelve Items Making Up the 
. Integration Subset within x^nalysis for Population B 

Response Rates Teacher 
Item Right Wrong Omit OTL First Part ofthe Item Stem 



1, 


76 


20 


4 


95 


/ (x - 1)^ dx is equal to . . . 


2. 


54 


36 


10 


3 


The line I in the figure is the graph oiy = fix). S f{x)cL 
equal to . . . 


3. 


23 


70 


7 


82 


The graph ofthe function fis shown above for 0 <.x <_ 1( 

/ fix) dx attains its greatest value . . . 
0 


4. 


36 


53 


11 


91 


The area enclosed between the curve y-x^-x^ and the 
T-axis is equal to . . . 


5. 


16 


59 


25 


58 


The value of dxix^ - 5x + 6) is . . . 

0 


6. 


63 


30 


7 


87 


At which point does the curve y = 3x^ - x^ have a local 
minimum? 


7. 


83 


15 


2 


94 


/ /x - 1 dx is equal to. . . 


8. 


19 


69 


12 


89 


Given that 3 /" ' (x) = x^ - 5, and fi2) = 1, then fiO) is 
equal to. . . 


9. 


56 


33 


11 


85 


(x - 1/x^) dx is equal to . . . 

1 


10. 


27 


50 


23 


62 


The function fis defined by fix) = f /I + du. f ' (2) 
equal to. . . ° 


11. 


25 


67 


8 


84 


Tjiis figure shows the graph oty = fix), . . . The value oi 

/ fix) dx is equal to . . . 
a 


12. 


31 


52 


18 


80 


12x/(2x + 1)^ dx is equal to . . . 

0 


Avg. 
Std.Err. 


42 
1.3 


46 
1.1 


12 
1.3 







mathematics every year, especially in the courses designed for specialists. An interesting finding was \ 
the proportion {Ticreased in two of the 15 countries for which data were available. The proper tiou decli 
in all others, however, xtioi a than 50 percent in Hong Kong and Japan. 

The proportion declined 22 percent in Ontario, close (as we would expect) to the 20 percent in Bri 
Columbia, but also close to the 19 percent decrease in the Flemish schools in Belgium. The USA was 
close at 16 percent. There was a greater decline of 27 percent in New Zealand and the Belgian Frc 
schools and still greater in England and Wales (37 percent) and Sweden (40 percent). Surprisin 
Scotland (at 7 percent) and Finland (at 10 percent) were quite different from their neighbors. In Israel, 
proportion of girls enrolled in mathematics classes declined only 13 percent from Population I 
Population B. 

With this sort of variation, one does not expect any simple explanation, and as usual this expecta 
proved correct. There was a strong link between the size of the group taking advanced mathematics 
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Table 3-6: Summary of Ontario Student Responses (percent) on the Sixteen Items 
Making Up the Algebra Subset within Analysis for Population B 



Response Rates 

Item Right Wrong Omit First Part of the Item Stem 



1. 


61 


36 


3 


Which of the following points lies in the region bounded by the line 
3^=1,^ = Xy and x + y = 6? 


2. 


78 


20 


2 


The curve defined by y = 3x(x - 2)(2x -h 1) intersects the x-axis only 
the points . . . 


3. 


31 


59 


10 


A stationer wants to make a card 8 cm long and of such a width thai 
when the card is cut into halves . . . 


4. 


12 


80 


8 


P is a polynomial in x of degree m, and Q is a polynomial in x of deg: 
with n<m. The degree of polynomial . . . 


5. 


72 


22 


2 


Which of the following, (x - 1), (x - 2), (x + 2), (x - 4) are factors of 
x^ - 4x^ - X + 4? 


6. 


21 


70 


9 


X and y are real numbers. The product of the matrices is commutat 
only if . . . 


7. 


15 


73 


11 


According to the graph, ox + 6 > co? when . . . 


8. 


74 


22 


4 


What are all values of x for which the inequality 5x + 5/3 <. - 2x - 2 
is true? 


9. 


78 


14 


8 


The equatioti of line Z is . . . and of line m is What is the solutio 

of the simultaneous equations ... 


10. 


55 


35 


10 


A certain number of students are to be accommodated in a hostel. 
If two students share each room, . . . 


11. 


54 


36 


10 


Two mathematical models are proposed to predict the return y, in 
dollars, from the sale of x thousand units of an article . . . 


12. 


44 


43 


13 


A piece of wire 52 cm long is cut into two parts and each part is ben 
to form a square. . . . How much longer . . . 


13. 


64 


34 


1 


If X is a real number, then defined by 3^ = /x^ - 1 , is a real numl 
for . . . 


14. 


46 


52 


1 


If X and y are real numbers, for which x car. you define 
ybyy = x//9-x^ ? 


15. 


45 


53 


2 


Given that a >^ 0, Vc^ is equal to : . . 


lb. 


00 




5 


A freight train traveling at 50 km/h leaves a station 3 h before an 
express train .... 


Avg. 
Std.Err. 


51 
0.9 


43 
0.8 


6 

0.5 
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take mathematics, also encourage girls-just not quite to the same extent. Looking at all 15 count 
however, there was no link between participation by girls in mathematics and participation by studen 
schooling in general. In other words, keeping more students in school does not necessarily resu 
keeping more girls in mathematics. Hints in the international analyses suggested a combinatic 
influences, including the amount and kind of mathematics covered. 

In Ontario, participation by girls was not the same in the three advanced courses offered in the ] 
pre-university year. The lowest participation was observed in the most demanding course, Grad 
algebra. This pattern was also observed in a 1983 science survey, which found higher participatio 
chemistry than in physics. Chemistry is a prerequisite for raany of the postsecondary program 
hea th-related professions, programs more often chosen, by girls. These same programs may encou 
applicants to take one Grade 13 mathematics course, and girls tend to choose relations and functior 
calculus rather than algebra. This is the pattern that leads to underrepresentation of women in sci, 
and engineering careers and to a very small proportion of women among mathematicians 



3.2.4. Semestered vs. Year-long Classes 



This leads us to an analysis of the mathematical learning experiences and accomplishments of 
whole group and how policies affect them. One of the many policies left to the local authority, or perhai 
the schools themselves, is the choice of full-year or semester organization. At the time SIMS was car 
out m Ontario, about a quarter of the secondary schools were organized on the semester system wl 
courses began in September and end in January, then begin again and end in June. In order to qualify 
credit, a course has to meet for at least 110 hours, so semester courses normally meet for double periods 
half of the year. Such a system is popular with students who want to hold jobs during the school year s 
timetables can be more flexible. Anecdotal reports suggest that the number of schools organizing on 
semester system is growing rapidly. 

A substudy was therefore made of the characteristics and outcomes in semester versus year-1 
schools. The study has been published (Raphael, Wahlstrom, & McLean, 1986), so only a brief summ 
will be given here. Contrary to assertions in the literature about the benefits of semester or^anizat 
achievement was higher in the year-long classes, and this difference could not be explained by ot 
characteristics of the classes and schools (such as social class, rural vs. urban, . . .). No differences « 
found m the students' attitude to mathematics, but teachers did report covering slightly more materia 
the year-long courses. 

Because SIMS was not designed for this comparison, the substudy has weaknesses that make 
results tentative. The design of the random sample of schools, for example, did not take semestei 
year-long organization into account, so the sample might not be representative of either More importt 
the achievement tests were given in May, by which time the year-long classes might have covered mor 
the syllabus than the half-year (semester) classes. Teachers did report, however, that thev had covered 
intended material and were already engaged in review in both types of classes by the time th^ SI 
booklets arrived. The result can give pause, therefore, to those who plan to change from year-lont 
semester organization. There are many reasons one might do so, but better mathematics achievemenl 
attitudes are not among them. 



-34- 



3.3. Linking SIMS Results Directly to Teaching and Learning 



The Population A and Population B samples of Ontario schools were carefully drawn to be representative 
of the province as a whole, using school size, region and type (separate, private, French, public). Such a 
process does not yield samples representative of any particular board of education. The boards, however, 
are the administrative units within which all educational decisions of consequence are taken, under the 
guidance of the Ministry of Education. However clear the results of the Ontario study are, teachers and 
officials in the local ^ufr.hority do not have the thorough sample of their classrooms that makes the results 
inescapably relevant to their particular board. 

A'c the encouragement of Steering Committee Chairman, Dr. H. Howard Russell, one board in 
Ontario repeated some of the SIMS study within their jurisdiction, administering the achievement 
booklets and the teacher OTL forms in all Grade 8 and Grade 13 classes. OISE provided copies of the 
booklets, analyzed the results and prepared a special report, under contract to the board. The outcomes 
were presented as school and classroom means, by achievement subset, coded so that local officials could 
identify schools and classes but others could not. The results were sufficiently useful that seven more 
boards have since done similar studies, not always in both populations. These local replications of SIMS 
are having profound effects throughout the mathematics programs of the boards conducting them. One 
board has now done the study in two school years. 

The pattern seems to be this. The results go to the Mathematics Coordinator and the Re;5earch 
Director (if any), who discuss them with serdor officials and Trustees (those elected to govern the 
authority). The Mathematics Coordinator then goes to every school, showing them where they stood 
relative to other schools in the board and to other countries. Officials have reported that they revised their 
teacher professional development plans to give aittention to problem areas identified in the study and made 
changes to their courses of study (the detailed version of Ministry guidelines at the school level). 

As the designers of SIMS expected, the description teachers and officials get of what they actually 
teach is itself of great interest, because they have never had such a systematic monitoring of their 
implemented curriculum. The officials have no trouble comparing this with their intended curriculum, 
which they know inside out. The achievement results usually confirm what teachers and officials 
expected, but the occasional surprise has again made the exercise worthwhile. A secondary school that 
had always been a leader with an excellent reputation failed to excel as they thought they should, bringing 
on a reappraisal of their program. All this has been possible, incidentally, without any publicity, giving 
the schools time to consider and act on the voluminous information without having to react to the 
oversimplified generalizations the media needs for its coverage. 
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Chapter 4 

Bringing Mathematics Teaching and Learning Together 



In previous chapters the study has been described-participants, curriculum analysis and attitudes. 
Results in Ontario were presented in some detail. In this chapter some results of particular interest are 
presented and their implications discussed in an international context. 

4.1. Ontario's Mathematics Program at Grade 8 

Recall that the definition of Population A in SIMS was "the grade in which the modal age is 13". There are 
two important research decisions lurking here. First, we have been concerned with the students in a 
particular grade of school-that is, with students in a particular set of classrooms, studying a particular 
program of mathematics. We are interested in how mathematics is taught and how it is learned. An 
alternative would be to study the achievements of the population of 13-year-olds, but that would mean, in 
most countries, sampling across perhaps three or four different grades, and the linkage to classroom 
instruction and curriculum would be lost. Second, the grade in which 13-year-olds are found is a point in 
the educational system of most countries when virtually all the age cohort is still in school, still taking 
mathematics and usually being given a common mathematics curriculum. Subsequently, students begin 
to stop taking mathematics, take different kinds of mathematics courses-or they stop going to school. 

The definition of Population A worked out well in Ontario, because Grade 8 is a point in the system 
when virtually all the age group attends school and takes the same mathematics course. In tha next grade 
in Ontario there is a major shift in mathematics education because most students transfer into secondary 
schools and begin to receive differentiated instruction in mathematics. After the next year the attrition 
from mathematics, and from school^ begins. 

The international perspective of SIMS shows us how Ontario's treatment of this critical juncture in 
mathematics education compares with what is done in other educational systems. It forces attention on 
some implicit and explicit curriculum choices and on the consequences of those choices. We will first focus 
on Ontario and the seven other countries that participated in the full SIMS Population A study, with 
beginning-of-year and end-of-year testing. 

4.1.1. Tracking and Streaming 

An important focus in the analysis of the international Population A data has been on the radically 
different approaches that countries make to the grouping of students for mathematics instruction. Some 
countries group, or stream, students by ability and others (Ontario, for example) have a policy against 
streaming. For the eight countries who tested at the beginning and end of the year it is possible to analyze 
the grouping practices empirically. Th»3 first technique to be used will be an examination of the variance in 
mathematics test scores (the spread or dispersion of the test scores) at the beginning of the school year, 
breaking that variance down into: 
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1. variance between schools-because students beginning Grade 8 in sorre schools score higher 
on tno average than students in other schools. This between-school variance might reflect 
duTerent neighborhood mixtures of students' socio-economic backgrounds, or it might 
correspond to quality differences in earlier mathematics programs (Grade 7 and before). 

2. variance between classrooms, wit'un schools-because right at the beginning of the school 
year, the mean mathematics achievement of classrooms normally range from high to low In 
countries where streaming is practiced at the Grade 8 level, this variance should be large, and 
It IS (in England, France and Hong Kong, for example). This variance should be low in 
Ontario, and it is, but not so low as in Japan. 

3. variance between students, within classrooms-because students within a classroom will 
mevitably show a great variation in their initial mathematics ability and accomplishment. 
This variance is high everywhere, even where streaming is practiced. 

Of particular interest is the proportion of variance that is found at each of the three levels-schools, 
classes and students. The three proportions of variaroe are called components. An analysis of the three 
components of variance was made on the one measure available from individual students-beginning-of- 
year scores on the core test. Recall that the core was an omnibus collection oi different mathematics topics 
and would not be useful in an analysis of classroom achievement. Some items were included because they 
were in the first study. It can serve to describe general mathematics background at the beginning of the 
year. 

The results are depicted in Figure 4-1, for which two technical points must be made. First, in four of 
the countries, only one classroom was sampled per school, so the estimates of school and classroom 
variance cannot be separated. Second, the variances have been estimated for true scores-scores for which 
the errors of measurement due to specific mathematics test item interactions have been discounted. 

Figure 4-1: Proportion of variance in pretest scores that can be 
attributed to schools, classrooms and students 
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The large v hite space in the middle of the bar shows that two of the countries. New Zealand and the 
USA, have made a clear educational choice in favor of differentiating students at this grade level. In the 
case of the USA, E. Kifer has studied how tracking determines and limits content exposure in the USA, 
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rurinai curriculum cunerentiation ^remedial, regular, enriched, and algebra classrooms may occur in the 
same school) largely determines content exposure and continuing mathematics opportunities. In Naw 
Zealand, the streaming is apparently more general and based on ability, but there is still a common 
curriculum. Japanese schools are extraordinary. Because of the sampling plan in Japan, school variance 
cannot be separated from classroom variance, but it is evident that both must be tiny. The schools and 
classrooms at this grade level in Japan are homogeneous in the extreme. The Belgium Flemish case is 
unusual in that they already have differentiated school systems at this grade le ^el-academic, vocational 
and the like. This results in large variation between schools. 

Ontario occupies a middle ground sc far as tracking and streaming goes. There is about 10 percent of 
the variance between schools, and another ID percent between classrooms within schools. It is education 
policy in Ontario that there should be no streaming or tracking in Grade 8, so the relatively small, yet 
statistically significant, amount of classroom variance indicates some departures from policy. One 
consequence of the relative lock of tracking may be that there is less opportunity for specialized instruction 
at either end of the ability continuum. Some argue that in a homogeneous system it is difiicult to provide 
special help and remediation for the poor mathematics students and the best mathematics students may be 
held back. When streaming begins in Grade 9, the difference is huge. A provincial survey (McLean, 
1982a) found that the difference in achievement between General and Advanced streams within Grades 9 
and 10 was greater than the gains in average achievement from Grade 7 to Grade 10. 

In spite of their successes, the Japanese are concerned about their schools and have launched a 
major reform effort. Bypassing the Ministry of Education, Prime Minister Nakasone named an Ad Hoc 
Commission on Educational Reform in 1984. The chairman is the former President of Kyoto University, 
and the membership is a cross section of Japanese society. In their second report, they found "rigidity, 
uniformity, and closedness ... a tendency to impose excessive controls on students". The system has 
"made wastelands of children's minds". Students are not taught to think independently; they are not 
allowed to develop "distinctive personalities or the ability to govern themselves". The Commission 
suggested very early that the qualities required of Japanese in the coming century should be 
broadmindedness, creativity, independent mind, and self-consciousness as a person in the international 
community.^^ It would appear that Japanese schools might not be so homogeneous in the next decade. 

The Ontario decision to have homogeneous mathematics instruction at Grade 8 was likely motivated 
more by social concerns than pedagogical ones. Does the apparent slippage into differentiation of 
classrooms within schools imply a perceived or practical difficulty of dealing with the full range of 
adolescent students in homogeneous classrooms? 



4.1.2. Content Differentiation 



The variance analysis in Figure 4-1 is based on pretest scores, that is, scores collected at the 
beginning of the school year and hence not influenced by instruction carried out during Grade 8. It 
therefore shows just the grouping practices. Now we will look at another way in which an educational 
system varies the mathematics learning opportunities of students-differentiation of the mathematics 
con/en^ presented during the school year. 

Information about content of instruction was collected from the teachers of the same classrooms for 
which the students were sampled. In Ontario this meant 197 teachers in 130 schools. As described in 
Section 2.5, each teacher was given, near the end of the school year, a complete set of achievement items 
(180) and asked to: 
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1. estimate w.iat percentage of the students would likely be able to answer the item correctly, 

2. indicate whether the "mathematical content necessary to answer the question" had been 
taught or reviewed during tne year, and 

3. indicate, in the case the content had not been taught or reviewed, whether it was taught in a 
previous year, a subsequent year, or not in the curriculum at all. 

These responses were used to calculate an opportunity to learn index (OTL), which in its simplest 
form indicates whether the teacher reports that an item was taught or reviewed or, il'not, whether it would 
Jiave been covered in an earlier year. A positive report means that the student had an opportunity to learn ' 
t'uti mathemati .! content and might be expected to get the right answer. The OTL variable can be 
averaged over all the items in a particular content area, such as arithmetic or algebra, and can be regarded 
as a characteristic of a classroom (how much OTL the students had) or even a province (how much algebra 
is taught in Ontario). 

In Figure 4-2 the distribution over classrooms of the OTL information of the classroom teachers is 
displayed, and this is differentiated both by the eight countries and by five principal content areas. Each 
distribution is shown as a boxplot: the box shows where the middle 50 percent of the classroom means are. 
with a bar at the median level; the lines go up and down to what we might regard as the "normal" extent of 
the distribution beyond the central part, and then the x's and o's mark extreme values. 

One finding evident in Figure 4-2 is that countries differ in the profiles of the mathematics content 
that students have (on the average) an opportunity to learn. Ontario, together with New Zealand and the 
USA, shows a lower opportunity to learn algebra, not a surprise since many algebra topics are optional at 
Grade 8. There is wide variation everywhere in the amount of geometry that is taught, and we know that 
this is complicated by the different kinds of geometry being taught, especially in Belgium and France, 
where many of the international test items were considered inappropriate. There is even more variation in 
the amount of teaching of statistics. For both geometry and statistics, Ontario is among the countries that 
teach rather more. In fact, we can see that with the exception of algebra, Ontario has one of the more 
extensive mathematics programs. 

The boxpiots also show the variation in OTL across classrooms. Except for arithmetic, the variance 
in OTL is generally high. For example, we can see that in Ontario, the geometry OTL is below about 30 
percent for a quarter of the classrooms and above about 60 percent for another quarter (these figures are 
marked by the top and bottom of the box, the 75th and 25th percent points, respectively). As noted in 
Chapter 2, this much variation is possible even when classrooms are all using the same mathematics 
curriculum because of optional topics and latitude for teachers to make choices. The question then arises, 
if we have a lot of classrooms with low coverage of geometry and algebra, what are they studying? Are 
they mostly reviewing arithmetic content from previous years? The SIMS design did not produce an 
answer to this question, since the focus was on the SIMS items rather than on the teachers' curriculum as a 
whole. 

4.1.3. Knowledge and Learning 

Analysis and interpretation of achievement findings has to be quite cautious in view of the 
international differences already evident in the composition of the mathematics classrooms and the 
curriculums as indicated by opportunity to learn. Technical difficulties also arise when we consider 
achievement results, because they are based ultimately on the test-taking abilities and practices of the 
students, and these are not even approximately constant across the eight countries. For example, the 
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percentage of omitted responses varies from 1 to 5 percent in Thailand to 5 to 50 percent in France 
Furthermore, we want to differentiate mathematical knowledge, which represents the accumulation of 
eight years of schooling, experience outside of school, common sense, and the like, from mathematical 
learning or growth which we can see by comparing achievement at the beginning and the end of the school 
year. 

One way to study growth is to look at (a) learners, students who did not know the answer to a 
question at the beginning of the .chool year but who did give the correct answer at the end of the year and 
(b) forgetters, students who answered correctly at the beginning but then answered incorrectly at the end of 
the year. In Figure 4-3, the learner and forgetter percentages, averaged over students and items are 
presented for each country and for three m^or content areas-arithmetic, geometry, and algebra The 
percentage of learners shows us in what countries there is evidence of growth, while the percentage of 
forgetters warns us of the possible effects of students getting the right answer by guessing. Forgetting 
might also be decline in achievement in older content, of course, or it might be mislearning. 

First, we see relatively little learning in arithmetic. As seen earlier, all countries reported very 
high OTL in arithmetic, but a more detailed examination of the OTL responses, and other responses by 
teachers to detailed questions about their mathematics instruction, revealed that the teaching in 
arithmetic for the most part is reviewing old content, that is, content such as fractions that had been taught 
earlier. 

Second, we see relatively little learning in geometry, with the notable exception of Japan But 
geometry again is difficult to interpret because the OTL is relatively low and the correspondence to 
national curriculum content is irregular. 

Finally, in algebra, we see a substantial amount of learning taking place in four countries, but not in 
Ontario. Ontario does not ask for much algebra, it does not stream students into algebra, and it does not 
get very much in the way of algebra achievement. In the light of the success that some other countries 
have m introducing algebra earlier, we should ask whether Ontario curriculum would be strengthened by 
enriching the algebra content in Grade 8. It is apparent that a good deal of time is spent, with little effect 
reviewing arithmetic skills such as addition of fractions that were covered in the previous year or over 
several previous years. 

4.2. Ontario's Advanced Senior Mathematics Program 

The definition of Population B, "students in the last year of secondary education who are taking advanced 
mathematics as a substantial (five hours a week) part of a program leading to post-secondary education- 
means that we now shift our attention from a universal mathematics program to an educational program 
for the mathematics elite. 

International comparisons in mathematics achievement at this level become especially difficult 
because there are enormous differences across the 12 countries for which data were collected in the nature 
of the student populations and, to some extent, in the content of the mathematics instruction While 
comparing achievement is difficult, the process is still worthwhile, because it call.«; our attention the 
crucial issue of how many students get how much mathematics. 
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Figure 4-3: Double barcharts of Percentage 

of Forgetters and Learners in Three Content areas 




42. 1. Selectivity and Specialization 

Since we are discussing the last year of secondary education, countries differ in the first instance in 
the proportion of the age cohort that survive to that level. Countries then differ again in the proportion of 
the surviving populations that can be considered to be mathematics specialists. This is depicted in Figure 
4-4, where the total age .cohort represents 100 percent, and we can see the proportion of mathematics 
specialists and the proportion of students in school. 

Fi^re 4-4: Barcharts of Population B Compared to Age and Grade Cohort 
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The full range of participation rates is evident across the 12 count-ies, from the very selective 
systems in England and New Zealand to the unselective systems in the U.S.A, Japan, and British 
Columbia. The Ontario data could be misleading for someone not familiar with the system, since the 
decision to include students taking two or more Grade 13 mathematics courses created a special subgroup 
of the university-bound population. Virtually all the students are from Grade 13, while the USA and 
British Columbia students are from Grade 12. 

In terms of mathematics specialization, considered as a percentage of the age cohort or as a 
percentage in the school grade, we see tremendous variation. In Hungary, only half the age cohort 
sur/ives to the final year of high school, but they all take advanced mathematics (which in Hungary means 
calculus). In British Columbia, a high percentage of the age cohort takes advanced mathematics, but as we 
shall see later, this does not include any calculus. In the other countries with high school participation 
rates, there is a compensating low mathematics participation rate. Ontario is again difficult to interpret 
and compare, but there is clearly a high rate of mathematics specialization. 

These data are from 1982-83, and with the new OAC regulations now going into effect one wonders 
what the effect will be on Ontario's international ranking in mathematics participation.^^ It is obvious 
that there are radically different philosophies and policies around the world about participation in senior 
secondary and about mathematics specialization, and this must have an influence on the technological 
depth and breadth of youth in society. 
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4.2.2. The Content of Senior Mathematics 

The mathematical content of the senior mathematics program is of interest mainly to mathematics 
educators, and particularly to the university educators who receive the students from the secondary 
schools. In Figure 4-5 we show the OTL results for analysis, which includes elementary functions and 
calculus. Note that this is based on students who are mathematics specialists, and the OTL results are not 
discounted for participation rates. 

Figure 4-5: Barcharts of Opportunity to Learn in Population B Mathematics 



U.S.A 



Ontario 
New Zealand 
Japan 
Israel 
Hungary 
Finland 
England/Wales 
Brit Columbia 
Belgium Fl. 




One hears complaints that entering university students are not adequately prepared in 
mathematics, that they need to relearn what they have stu'iied in high school. A particular issue in 
mathematics is the utility and efficacy of the instruction in calculus. In Ontario, virtually all mathematics 
specialists take calculus in high school. This is also true in most other countries, with the notable 
exceptions of the USA and British Columbia. It is perhaps appropriate to consider those exceptions in light 
of the revisions that have now been made in Ontario's mathematics program. In British Columbia, 
calculus is simply not available to students, and the senior mathematics program is a rather thorough 
course in algebra and trigonometry. The universities expect to teach calculus. In the USA, there is an 
unusual national program in advanced placement calculus; this is offered by some schools to their best 
students, and with a national testing program, universities gr^nt advanced placement to successful 
students. Ontario students study calculus in Grade 13 and then study it again in first-year university. 
Their performance was poor on the SIMS trigonometry item subset. 

Is it really appropriate to have so much emphasis on calculus in a system that has such a high 
participation rate? It has been argued that this is the reason calculus is retaught in the universities, and 
that time could be better spent in the high schools consolidating pre-calculus mathematics or on other 
topics altogether. The Project Committee drawing up the new mathematics curriculum guideline has 
responded to this argument and to the SIMS results by creating new courses (see Note 13), with some 
topics shifted and given more emphasis. Trigonometry, for example, has been moved from the relations 
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and functions course (which will be dropped) and made a core topic in calculus. There is a new course, 
firiite mathematics, covering matrices, combinatorics, and probability and statistics. 



4.2.3. Training the Mathematics Elite 

The large differences in participation rates and content of mathematics instruction make it 
impossible to do sensible international comparisons of achievement related to instruction. If you try to 
educate a large fraction of the age cohort, it seems inevitable that the average achievement will fall. We 
can still make comparisons, however, among the very best mathematics students. Prosumably, every 
country strives to give adequate opportunities for these students to learn mathematics, and it is well 
known that high-level mathematical ability often emerges at a young age. 

In a complex analysis made across the 12 countries (Miller & Linn, 1985), a scoring system was 
devised and then used in combination with the age cohort participation rates to estimate three scores: 

• the mean score in analysis, based on all the students who took the test, 

• the test score corresponding to the top 5 percent of the age cohort, and 

• the test score corresponding to the top 1 percent of the age cohort. 

Although the first score is very much a function of the participation pattern in the countries, it can 
be argued that the latter two scores are comparable across countries. We have to assume that the top 
participants score higher than any non-participants. The results of this analysis are depicted in Figure 
4-6. < 

Figure 4-6: Line Chart of Mean Population B Achievement Levels in Analysis 
and Estimated Achievements of Top Percents of Age Cohort 
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As expected, lower mean scores were found in the countries with higher participation rates, but the 
country-to-country ranking for the 1st and 5th percentile scores gives quite a different story. Clearly, 
Japan wins the elite competition; the top 5 percent of its students do better than the top 1 percent of 
students anywhere else. Ontario's elite does very well, comparable to the elite in, for exarr.ple. New 
Zealand, which is more selective, and to Hungary, which is less selective. In fact, there seems to be little 
connection between the selectivity of the system and the performance of the elite, 
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4,3. Summary and Conclusions 

Ontario's participation in the Second International Mathematics Study has yielded an amazingly detailed 
monitoring of the Grade 8 and the Grade 12/13 mathematics program. Availability of the international 
results has now made it possible to produce this first public report, a very small sample of the results and 
implications available. Curriculum committees in the Ministry of Education have had access to the full 
report for some time and have made use of it in guideline revisions. One hope is that this sample will 
encourage more mathematics educators to dig into the full report. Better still, with OISE's help they can 
dig into the data to answer questions the OISE team did not address. 

As noted at the beginning of this report, the evidence is strong that Ontario schools could offer more 
mathematics content than they do at the Grade 8 level and that a large majority of the students could learn 
and profit from it. There is evidence from the variation among classes that some schools may already be 
doing this, in opposition to the spirit, if not the letter, of provincial regulations. The very considerable 
amoimts of time spent reviewing and reteaching some arithmetic concepts does not yield results 
proportional to the time spent, so some change should be made. Either better methods have to be found or 
else the time should be spent on other topics where more progress can be made. The following quotation 
from the new mathematics guideline for Grade 8 (section on Whole Numbers undDecimals) illustrates that 
this message is getting through (Ministry of Education, 1985). 

The work on place value consolidates and extends that of Grade 7. Computational skills should 
be reviewed and practised in the setting of applications and problem solving with an appropriate 
use of estimation and mental computation to anticipate the reasonableness of results (p. 34). 

The general area of algebra was suggested for more attention, though this need not be abstract and 
formal treatment. Working through realistic problems involving money and measurement, using 
calcuJators and computers, can bring in algebra concepts and still give practice on arithmetic operations. 
Drill on arithmetic worksheets devoid of any real world context might well be banned from the 
intermediate division. Here is the introduction to the section. Variables, Formulas, and Equations, in the 
new guideline. 

Number patterns, arithmetic problems, and measurement formulas should be used to review the 
use of algebraic notation. The emphasis should be placed on the interpretation of algebraic 
expressions and equations as generalizations of arithmetical expressions and equations. 
Informal but systematic methods of solving equations should be consolidated (p. 36). 

There was good news in the fine performance of Ontario's top students. The top 5 percent and 1 
percent do well in comparison to other countries, in a system where a high proportion of the age group is in 
school and a high proportion of those takes mathematics. The SIMS achievement items were not such as to 
stretch the very best students to the fullest (in any country), but it was encouraging, nevertheless, that 
Ontario can provide opportunity to many without disadvantaging the top few. 

That said, there are reasons to be wary of the future. Grade 13 students learned a lot of very 
elementary calculus that they may have to go over again in university, and they achieved only mediocre 
results in pre-calculus topics such as trigonometry. It remains to be seen whether a slight shift in the 
provincial guideline can bring about measurable improvement. Ontario students did well in algebra. 
\Vhat will happen with the demise of Grade 13 and the advent of the OACs? If the Calculus remains 
elementary and thrives and the new Algebra and Geometry withers, it would not bode well. 

The OACs are said to be advanced courses, and in many respects they are. An advanced topic in the 
intended curriculum but absent from both the implemented and achieved was complex numbers. Every 
teacher said the topic was taught in someone else's class, and the students said they had never heard of 



-46- 



complex numbers. In the new guideline, Complex Numbers is a core topic in Algebra and Geometry-and 
nowhere else. At least the intended curriculum is clear. 

Finally. SIMS taught us a lot about how we should study mathematics education. Attention to the 
three curriculums proved to be a major strength. Never again should we be content with just the achieved 
curriculum. The longitudinal study in Population A (beginning- and end-of-year testing) brought home the 
necessity for measures of growth if ;ve are to relate achievement to teaching. A test score at one point in 
time IS hopelessly confounded with all prior learning, even for new topics, and such a score tells us little or 
nothing we can use to improve teaching. A good example was the Ontario Mathematics Achievement Test 
(OMAT), included as a national option because there were scores available going back to 1968 It was 
encouraging to learn that the average score had remained constant over 15 years (Wahlstrom, Raphael, & 
McLean, 1986), but beyond that it seemed to have little to tell us. 

SIMS relied entirely on multichoice achievement items, at least partly because it was felt no other 
type could be used in a comparable way internationally. There were enough prt blems with the multichoice 
Items to make one question their practical superiority but, more than that, tl.oir substantive limitaiions 
are just too great. If we are to learn whether students can solve problems, we have to give them some 
problems to solve and not give them a few answers from which to choose. Even a small number of 
constructed-response items would have added enormously to our understanding of the achieved 
curriculum internationally and nationally. 

Ontario has already profited from its participation in SIMS-the separate reports have b-en 
discussed and the materials have been used to good effect at the local and provincial levels. Seven boards 
have repeated a large part of the study in their jurisdiction, with perhaps more to come. The survey of 
mathematics as actually taught and the achievement results closely tied to the teaching have been studied 
carefully and used to make changes in the official intended curriculum and locally in professional 
development. With the publication of this report, a larger audience can see what has been achieved. 
Perhaps the valuable outcomes are just beginning to appear. 
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Appendix A 

Participants in the Second International 
Mathematics Study 



"Country" 
Australia 



Degree of Participation 
Replicated First Study 



Extras 



Belgium (Flemish schools) Pop. A Longitudinal 

Pop. B Posttest 

Belgium (French schools) Pop. A and B Posttest 

Canada (British Columbia) Pop. A: core pretest -f posttest 

Pop. B: posttest only 



Canada (Ontario) 
Chile 

Dominican Republic 

England and Wales 

Finland 

France 

Hong Kong 

Hungary 

Israel 

Ireland 

Ivory Coast 

Japan 

Luxembourg 
Netherlands 



Pop. A, Longitudinal 
Pop. B, posttest only 

Withdrew before testing 

Pop. A, longitudinal 
(started late--not in 
international reports) 

Pop. A and B Posttest 

Pop. A and B Posttest 

Pop. A Longitudinal 

Pop. A and B Posttest 

Pop. A and B Posttest 

Pop. A and B Posttest 

Curriculum analysis only 

Withdrew, no data 

Pop; A Longitudinal 
(special pretest) 
Pop.B Posttest 

Pop.A Posttest 

Pop. A Posttest 



None 



Pop. A: Small group 
given posttest only as control. 
Pop. B: Ont. Math. Aptitude 
Test added. 
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New Zealand 

Nigeria 

Scotland 

Swaziland 

Sweden 

Thailand 

United States of America 



Pop. A Longitudinal 
Pop. B Posttest 

Pop. A Posttest 

Pop. A and B Po5>ttest 

Pop. A Posttop.t 

Pop. A and B Posttest 

Pop. A Longitudinal 
Pop. B Posttest 

Pop. A: Longitudinal 
Pop. B: Longitudinal 



Posttest only 
control sample 
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Appendix B 

Reports Published or To Be Published on the 
Second International Mathematics Study 



International Volumes 

Results will be published in three volumes by Pergamon Press in 1987. 

Travers, Kenneth J. & Westbury, Ian. Volume I: Analysis o/' the International Mathematics 
Curriculum. 

Garden, Robert A. & Robitaille, David F. Volume II: Student Achievement in Twenty-Two Countries. 

Burstein, Leigh Schwille, John, Cooney, Thomas J., Robin, Daniel, Robitaille, David F & Travers 
Kenneth J. Volume III: Student Growth and Classroom Processes in Lower Secondary School. ' 

Selected National Reports 

Crosswhite F Joe, Dossey, John A., Swafford, James 0., McKnight, Curtis, C. & Travers, Kenneth 
J (1985) Second International Mathematics Study: Summary Report for the United States 
Champaign, Illinois: Stipes Publishing Company. 

National Institute for Educational Research. National Report of Japan 

(in Japanese. A very short English summary is available for each volume): 

• Mathematics Achievement of Secondary School Students-Second International 
Mathematics Study. Volume I, September, 1 981. 

• Mathematics Achievement and Associated Factors of Secondary School Students- 
Second International Mathematics Study. Volume II, March 1982. 

• Mathematics Achievement and Teaching Practice in Lower Secondary Schools (Grade 
7) -Second International Mathematics Study. Volume III, March 1983. 

Robitaille David F O'Shea, Thomas J. & Dirks, Michael K. (1982) The Second International 
Mathematics Study: The Teaching and Learning of Mathematics in British Columbia Victoria 
B.C.: British Columbia Ministry of Education, Learning Assessment Branch. 

Ontario R eports on SIMS 

McLean, Leslie D., Raphael, Dennis & Wahlstrom, Merlin W. (1986) Intentions and Attainments in 
if ^ n'^ and Learning of Mathematics-Report on the Second International Mathematics 
btudy in Ontario, Canada. Toronto: Ontario Ministry of Education. 

Raphael, D., Wahlstrom, M. W., & McLean, L. D. (in press). School structure and its relationship to 
student attitudes and achievement in mathematics. International Review of Education. 

Raphael D. & Wahlstrom, M.W. (in press) Use of instructional aids in mathematics teaching- the 
influence upon achievement. Journal for Research in Mathematics Education. 
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Notes 



A list of Ontario and International reports is presented in Appendix B. Readers wanting a full 
account of the study will want to consult the international volumes or the national reports Appendix A 
contains a complete list of participants and the extent of their participation. Only a brief summary of 
findings IS presented here, the primary purpose being to put the Ontario results in international 
perspective. 

^Canada chose not to participate in the planning and implementation of the first study. The reasons 
appeared to be more social and political than any judgement of the quality of the plan. 

^Swaziland's average age was 15.7 and Nigeria's 16.7, owing to special circumstances that result in 
many older students attending elementary school. 

^Occupations were coded using the Pineo-Porter scale (Pineo, Porter, & McRoberts, 1977) For 
details, see the Ontario national report (McLean, Raphael, & Wahlstrom, 1986, Appendix' VI) For the 
international analyses, adjustments were made to the scales in an attempt to make them more 
comparable. These adjustments resulted in some differences between the international and the national 
percentages. In Ontario Table 2-1, for example, the percentage of fathers in unskilled and semi-skilled jobs 
was given as 30 percent, whereas the international report sets the Figure at 36.6 percent The 30 Dercent 
would still be high. f ■ i-cui, 

^Eligible students are those meeting the definition of Pop. B. In every country but one (Hungary) 
the retention rate was lower than the percentage of the age group still in school. The lower figure was used 
m all calculations involving retention rate in the SIMS study. 

^Mathematics Retention Rate is Population B as a percent of the total age cohort. Mathematics 
Participation Rate is Population B as a percent, of the grade cohort. General Retention Rati is the grade 
cohort as a percent of the age cohort. 

■^The synthesis was done by a Harvard Researcher, Helen Featherstone, and reported in the Times 
(London) Educational Supplement. 

*rhis is something of a tautology, since the mathematics under consideration might well be called 
school mathematics-taught and used in school but rarely used anywhere else by most people. This is in 
contrast to folk mathematics, techniques for solving numerical and measurement problems by means not 
taught in school but learned and used in practice. 

^'One of 14 pages in the Algebra classroom processes questionnaire. Teachers were asked on page 
ont whether the topic was (a) Taught as new content, (b) Reviewed and then extended, (c) Reviewed only 
(d) Assumed as prerequisite knowledge and neither taught nor reviewed, or (e) Not taught and not 
assumed as prerequisite knowledge. 

^^Items had a mean difficulty across countries in the interval 0.4 to 0.9, a range of difficulties across 
systems and a mean discrimination of at least 0.3. Items of particular interest in this study were those 
J"^g!^ ^^^^^y to show growth over the year and those testing computational skills, estimation and 
awp^ -'im U'onvprQportional reasoning, items for which a calculator would be useful and "new math" items 
fffva^fir-t midil Ahvr items were included, chiefly linear algebra and geometry, because the Belgian 
^"Hmf^p mrrsmm^eh said these aresiS were important to their curriculums and were not adequately tested 
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'^ISewsleUer of the National Institute for Educational Research, Vol. 18, No. 1 (March 1986), page 4. 

^^For readers outside Ontario, OAC stands for Ontario Academic Course, courses which are part of a 
plan introduced in 1985 that will enable some students to complete secondary school and qualify for 
university in four years rather than f: ve. Initially, most Grade 13 courses will be given as OACs, including 
the three mathematics courses-Algebra, Calculus and Relations and Functions. New courses are being 
developed, and by 1989 the three mathematics courses will be Algebra and Geometry, Calculus, and Finite 
Mathematics. 
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